Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 851683 Details for
Bug 1039115
Web Admin unresponsive after loss of one dns domain server
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
Most recent crash. Took 12 hours to get back up.
12hourhazing.txt (text/plain), 1.75 MB, created by
Ryan Womer
on 2014-01-17 16:06:14 UTC
(
hide
)
Description:
Most recent crash. Took 12 hours to get back up.
Filename:
MIME Type:
Creator:
Ryan Womer
Created:
2014-01-17 16:06:14 UTC
Size:
1.75 MB
patch
obsolete
>2014-01-09 12:25:12,633 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-11) Attempting to update VMs/Templates Ovf. >2014-01-09 12:25:12,635 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-11) Attempting to update VM OVFs in Data Center CSEnt >2014-01-09 12:25:12,636 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-11) Successfully updated VM OVFs in Data Center CSEnt >2014-01-09 12:25:12,637 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-11) Attempting to update template OVFs in Data Center CSEnt >2014-01-09 12:25:12,638 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-11) Successfully updated templates OVFs in Data Center CSEnt >2014-01-09 12:25:12,639 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-11) Attempting to remove unneeded template/vm OVFs in Data Center CSEnt >2014-01-09 12:25:12,640 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-11) Successfully removed unneeded template/vm OVFs in Data Center CSEnt >2014-01-09 12:25:23,005 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-27) Start refreshing all users data >2014-01-09 12:30:17,889 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetIsoListVDSCommand] (pool-6-thread-47) START, GetIsoListVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false), log id: 1f649efd >2014-01-09 12:30:17,890 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HsmGetIsoListVDSCommand] (pool-6-thread-47) START, HsmGetIsoListVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 7020425 >2014-01-09 12:30:17,923 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HsmGetIsoListVDSCommand] (pool-6-thread-47) FINISH, HsmGetIsoListVDSCommand, return: [virtio-win-0.1-52.iso, centos64x64netinst.iso, virtio-win-0.1-65.iso, winserve2012.iso, kali-linux-1.0.5-amd641.iso, Server2008R2SP1-en-US-July2013.iso, win7.iso, virt-p2v-0.9.1-2.20130730.1.el6.centos.iso, zentyal-3.2-amd64.iso, Elastix-2.4.0-Stable-x86_64-bin-04feb2013.iso, EXCH201064.ISO], log id: 7020425 >2014-01-09 12:30:17,925 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetIsoListVDSCommand] (pool-6-thread-47) FINISH, GetIsoListVDSCommand, return: [virtio-win-0.1-52.iso, centos64x64netinst.iso, virtio-win-0.1-65.iso, winserve2012.iso, kali-linux-1.0.5-amd641.iso, Server2008R2SP1-en-US-July2013.iso, win7.iso, virt-p2v-0.9.1-2.20130730.1.el6.centos.iso, zentyal-3.2-amd64.iso, Elastix-2.4.0-Stable-x86_64-bin-04feb2013.iso, EXCH201064.ISO], log id: 1f649efd >2014-01-09 12:30:17,931 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (pool-6-thread-47) Finished automatic refresh process for ISO file type with success, for storage domain id de2fccd8-63cc-4662-8878-077b4cfb810e. >2014-01-09 12:51:36,074 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (ajp--127.0.0.1-8702-5) [4112e11f] Lock Acquired to object EngineLock [exclusiveLocks= key: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 value: VDS_FENCE >, sharedLocks= ] >2014-01-09 12:51:36,075 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (pool-6-thread-50) [4112e11f] Running command: FenceVdsManualyCommand internal: false. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 12:51:36,080 INFO [org.ovirt.engine.core.bll.ClearNonResponsiveVdsVmsCommand] (pool-6-thread-50) Running command: ClearNonResponsiveVdsVmsCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 12:51:36,084 INFO [org.ovirt.engine.core.vdsbroker.UpdateVdsVMsClearedVDSCommand] (pool-6-thread-50) START, UpdateVdsVMsClearedVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8), log id: 324cf699 >2014-01-09 12:51:36,086 INFO [org.ovirt.engine.core.vdsbroker.UpdateVdsVMsClearedVDSCommand] (pool-6-thread-50) FINISH, UpdateVdsVMsClearedVDSCommand, log id: 324cf699 >2014-01-09 12:51:36,090 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-50) Correlation ID: 27753787, Call Stack: null, Custom Event ID: -1, Message: All VMs' status on Non Responsive Host ovirt01 were changed to 'Down' by ryan-a >2014-01-09 12:51:36,097 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-50) Correlation ID: 4112e11f, Call Stack: null, Custom Event ID: -1, Message: Manual fence for host ovirt01 was started. >2014-01-09 12:51:36,098 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (pool-6-thread-50) Lock freed to object EngineLock [exclusiveLocks= key: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 value: VDS_FENCE >, sharedLocks= ] >2014-01-09 12:51:40,001 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp--127.0.0.1-8702-11) [41c00f9f] Lock Acquired to object EngineLock [exclusiveLocks= key: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 value: VDS >, sharedLocks= ] >2014-01-09 12:51:40,012 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-6-thread-47) [41c00f9f] Running command: ActivateVdsCommand internal: false. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 12:51:40,013 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-6-thread-47) [41c00f9f] Before acquiring lock in order to prevent monitoring for host ovirt01 from data-center CSEnt >2014-01-09 12:51:40,014 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-6-thread-47) [41c00f9f] Lock acquired, from now a monitoring of host will be skipped for host ovirt01 from data-center CSEnt >2014-01-09 12:51:40,016 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-6-thread-47) [41c00f9f] START, SetVdsStatusVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, status=Unassigned, nonOperationalReason=NONE), log id: 4618a13c >2014-01-09 12:51:40,019 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-6-thread-47) [41c00f9f] FINISH, SetVdsStatusVDSCommand, log id: 4618a13c >2014-01-09 12:51:40,021 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (pool-6-thread-47) [41c00f9f] START, ActivateVdsVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8), log id: 67e226db >2014-01-09 12:51:40,889 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (pool-6-thread-47) [41c00f9f] START, GetHardwareInfoVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01]), log id: 7d3baacc >2014-01-09 12:51:41,066 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (pool-6-thread-47) [41c00f9f] FINISH, GetHardwareInfoVDSCommand, log id: 7d3baacc >2014-01-09 12:51:41,076 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (pool-6-thread-47) Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 12:51:41,079 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (pool-6-thread-47) Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 12:51:41,081 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (pool-6-thread-47) FINISH, ActivateVdsVDSCommand, return: Host[ovirt01], log id: 67e226db >2014-01-09 12:51:41,182 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-6-thread-47) Activate finished. Lock released. Monitoring can run now for host ovirt01 from data-center CSEnt >2014-01-09 12:51:41,185 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-47) Correlation ID: 41c00f9f, Job ID: fb21f495-02f6-42f1-8c43-4c4ea31fbfb7, Call Stack: null, Custom Event ID: -1, Message: Host ovirt01 was activated by ryan-a. >2014-01-09 12:51:41,187 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-6-thread-47) Lock freed to object EngineLock [exclusiveLocks= key: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 value: VDS >, sharedLocks= ] >2014-01-09 12:51:41,277 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-18) START, GetHardwareInfoVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01]), log id: 5a78f515 >2014-01-09 12:51:41,287 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-18) FINISH, GetHardwareInfoVDSCommand, log id: 5a78f515 >2014-01-09 12:51:41,299 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-18) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host ovirt01. >2014-01-09 12:51:41,308 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-18) Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 12:51:41,312 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-18) Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 12:51:42,118 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-18) START, ConnectStorageServerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = ISCSI, connectionList = [{ id: 623aa506-12d1-4437-8e81-f39b2317f3dd, connection: 192.168.101.101, iqn: iqn.2013-11.local.cytechservices.n16000pro:iscsi.hosts-tgt00.raid0, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 22f6bd5f >2014-01-09 12:51:45,101 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-18) FINISH, ConnectStorageServerVDSCommand, return: {623aa506-12d1-4437-8e81-f39b2317f3dd=0}, log id: 22f6bd5f >2014-01-09 12:51:45,102 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-18) Host ovirt01 storage connection was succeeded >2014-01-09 12:51:45,104 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-18) START, ConnectStorageServerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 210997b7-9098-48d6-8e97-9689085aa4b0, connection: 192.168.101.101:/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: AUTO, nfsRetrans: null, nfsTimeo: null };]), log id: 78ed04b3 >2014-01-09 12:51:45,447 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-18) FINISH, ConnectStorageServerVDSCommand, return: {210997b7-9098-48d6-8e97-9689085aa4b0=0}, log id: 78ed04b3 >2014-01-09 12:51:45,448 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-18) Host ovirt01 storage connection was succeeded >2014-01-09 12:51:45,450 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-18) START, ConnectStorageServerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 4e5d4030-1bd2-4487-b08f-2135eba3f11f, connection: 192.168.101.101:/vms, iqn: null, vfsType: null, mountOptions: null, nfsVersion: V4, nfsRetrans: null, nfsTimeo: null };]), log id: 5474cd9f >2014-01-09 12:51:45,529 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-18) FINISH, ConnectStorageServerVDSCommand, return: {4e5d4030-1bd2-4487-b08f-2135eba3f11f=0}, log id: 5474cd9f >2014-01-09 12:51:45,530 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-18) Host ovirt01 storage connection was succeeded >2014-01-09 12:51:45,534 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-6-thread-50) START, ConnectStoragePoolVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, vds_spm_id = 1, masterDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, masterVersion = 2), log id: 3904056d >2014-01-09 12:51:55,551 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-6-thread-50) FINISH, ConnectStoragePoolVDSCommand, log id: 3904056d >2014-01-09 12:51:55,567 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-18) START, SetMOMPolicyParametersVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8), log id: e427ce8 >2014-01-09 12:51:55,645 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-18) FINISH, SetMOMPolicyParametersVDSCommand, log id: e427ce8 >2014-01-09 12:51:55,649 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-18) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host ovirt01. >2014-01-09 12:51:55,653 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-18) Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 12:51:55,656 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-18) Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 12:52:12,318 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-10) [676c3913] Lock Acquired to object EngineLock [exclusiveLocks= key: be5348a1-85c7-48dd-a8ad-8a0464643bd8 value: VM >, sharedLocks= ] >2014-01-09 12:52:12,334 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-10) [676c3913] START, IsVmDuringInitiatingVDSCommand( vmId = be5348a1-85c7-48dd-a8ad-8a0464643bd8), log id: 772c6fe3 >2014-01-09 12:52:12,335 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-10) [676c3913] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 772c6fe3 >2014-01-09 12:52:12,362 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-6-thread-47) [676c3913] Running command: RunVmCommand internal: false. Entities affected : ID: be5348a1-85c7-48dd-a8ad-8a0464643bd8 Type: VM >2014-01-09 12:52:12,381 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-6-thread-47) [676c3913] START, IsoPrefixVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 109bd442 >2014-01-09 12:52:12,382 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-6-thread-47) [676c3913] FINISH, IsoPrefixVDSCommand, return: /rhev/data-center/mnt/192.168.101.101:_iso/de2fccd8-63cc-4662-8878-077b4cfb810e/images/11111111-1111-1111-1111-111111111111, log id: 109bd442 >2014-01-09 12:52:12,386 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-6-thread-47) [676c3913] START, CreateVmVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, vm=VM [Apache01]), log id: 7ebd77f >2014-01-09 12:52:12,393 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-47) [676c3913] START, CreateVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, vm=VM [Apache01]), log id: 2f1135fb >2014-01-09 12:52:12,457 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-47) [676c3913] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand spiceSslCipherSuite=DEFAULT,memSize=8192,kvmEnable=true,smp=4,vmType=kvm,emulatedMachine=rhel6.4.0,keyboardLayout=en-us,memGuaranteedSize=2048,nice=0,display=qxl,smartcardEnable=false,smpCoresPerSocket=1,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,timeOffset=0,transparentHugePages=true,vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8,devices=[Ljava.util.HashMap;@2b4da48,acpiEnable=true,vmName=Apache01,cpuType=Nehalem,custom={device_2528f132-6ebf-4acb-8b86-aecf0bc31cc3device_ffc79dbb-122a-4ad9-a297-f0857192f939device_6ad955b3-e66b-4f0a-9639-aac0feba5432device_f098485f-6878-4242-8c95-e8d69712107a=VmDevice {vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, deviceId=f098485f-6878-4242-8c95-e8d69712107a, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}}, device_2528f132-6ebf-4acb-8b86-aecf0bc31cc3device_ffc79dbb-122a-4ad9-a297-f0857192f939device_6ad955b3-e66b-4f0a-9639-aac0feba5432device_f098485f-6878-4242-8c95-e8d69712107adevice_1f6ab695-a846-48df-91c1-452cb29350b7=VmDevice {vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, deviceId=1f6ab695-a846-48df-91c1-452cb29350b7, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}}, device_2528f132-6ebf-4acb-8b86-aecf0bc31cc3device_ffc79dbb-122a-4ad9-a297-f0857192f939device_6ad955b3-e66b-4f0a-9639-aac0feba5432=VmDevice {vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, deviceId=6ad955b3-e66b-4f0a-9639-aac0feba5432, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}}, device_2528f132-6ebf-4acb-8b86-aecf0bc31cc3=VmDevice {vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, deviceId=2528f132-6ebf-4acb-8b86-aecf0bc31cc3, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}}, device_2528f132-6ebf-4acb-8b86-aecf0bc31cc3device_ffc79dbb-122a-4ad9-a297-f0857192f939=VmDevice {vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, deviceId=ffc79dbb-122a-4ad9-a297-f0857192f939, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}}} >2014-01-09 12:52:12,466 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-47) [676c3913] FINISH, CreateVDSCommand, log id: 2f1135fb >2014-01-09 12:52:12,470 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-6-thread-47) [676c3913] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 7ebd77f >2014-01-09 12:52:12,470 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-6-thread-47) [676c3913] Lock freed to object EngineLock [exclusiveLocks= key: be5348a1-85c7-48dd-a8ad-8a0464643bd8 value: VM >, sharedLocks= ] >2014-01-09 12:52:12,473 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-47) [676c3913] Correlation ID: 676c3913, Job ID: 731719a1-4ffc-4610-a7dc-4996a075d1c6, Call Stack: null, Custom Event ID: -1, Message: VM Apache01 was started by ryan-a (Host: ovirt02). >2014-01-09 12:52:16,729 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-81) START, DestroyVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, force=false, secondsToWait=0, gracefully=false), log id: 744a035d >2014-01-09 12:52:17,291 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-81) FINISH, DestroyVDSCommand, log id: 744a035d >2014-01-09 12:52:17,299 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-81) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM Apache01 is down. Exit message: internal error process exited while connecting to monitor: Failed to allocate 8589934592 B: Cannot allocate memory >. >2014-01-09 12:52:17,300 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-81) Running on vds during rerun failed vm: null >2014-01-09 12:52:17,301 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-81) vm Apache01 running in db and not running in vds - add to rerun treatment. vds ovirt02 >2014-01-09 12:52:17,306 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-81) Rerun vm be5348a1-85c7-48dd-a8ad-8a0464643bd8. Called from vds ovirt02 >2014-01-09 12:52:17,309 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-47) Correlation ID: 676c3913, Job ID: 731719a1-4ffc-4610-a7dc-4996a075d1c6, Call Stack: null, Custom Event ID: -1, Message: Failed to run VM Apache01 on Host ovirt02. >2014-01-09 12:52:17,315 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-6-thread-47) Lock Acquired to object EngineLock [exclusiveLocks= key: be5348a1-85c7-48dd-a8ad-8a0464643bd8 value: VM >, sharedLocks= ] >2014-01-09 12:52:17,329 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (pool-6-thread-47) START, IsVmDuringInitiatingVDSCommand( vmId = be5348a1-85c7-48dd-a8ad-8a0464643bd8), log id: 9c18ded >2014-01-09 12:52:17,330 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (pool-6-thread-47) FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 9c18ded >2014-01-09 12:52:17,349 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-6-thread-47) Running command: RunVmCommand internal: false. Entities affected : ID: be5348a1-85c7-48dd-a8ad-8a0464643bd8 Type: VM >2014-01-09 12:52:17,366 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-6-thread-47) START, IsoPrefixVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 34083655 >2014-01-09 12:52:17,367 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-6-thread-47) FINISH, IsoPrefixVDSCommand, return: /rhev/data-center/mnt/192.168.101.101:_iso/de2fccd8-63cc-4662-8878-077b4cfb810e/images/11111111-1111-1111-1111-111111111111, log id: 34083655 >2014-01-09 12:52:17,370 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-6-thread-47) START, CreateVmVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, vm=VM [Apache01]), log id: 226d52fc >2014-01-09 12:52:17,373 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-47) START, CreateVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, vm=VM [Apache01]), log id: 7dcba62c >2014-01-09 12:52:17,439 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-47) org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand spiceSslCipherSuite=DEFAULT,memSize=8192,kvmEnable=true,smp=4,vmType=kvm,emulatedMachine=rhel6.4.0,keyboardLayout=en-us,memGuaranteedSize=2048,nice=0,display=qxl,smartcardEnable=false,smpCoresPerSocket=1,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,timeOffset=0,transparentHugePages=true,vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8,devices=[Ljava.util.HashMap;@40f5250c,acpiEnable=true,vmName=Apache01,cpuType=Nehalem,custom={device_2528f132-6ebf-4acb-8b86-aecf0bc31cc3device_ffc79dbb-122a-4ad9-a297-f0857192f939device_6ad955b3-e66b-4f0a-9639-aac0feba5432device_f098485f-6878-4242-8c95-e8d69712107a=VmDevice {vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, deviceId=f098485f-6878-4242-8c95-e8d69712107a, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}}, device_2528f132-6ebf-4acb-8b86-aecf0bc31cc3device_ffc79dbb-122a-4ad9-a297-f0857192f939device_6ad955b3-e66b-4f0a-9639-aac0feba5432device_f098485f-6878-4242-8c95-e8d69712107adevice_1f6ab695-a846-48df-91c1-452cb29350b7=VmDevice {vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, deviceId=1f6ab695-a846-48df-91c1-452cb29350b7, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}}, device_2528f132-6ebf-4acb-8b86-aecf0bc31cc3device_ffc79dbb-122a-4ad9-a297-f0857192f939device_6ad955b3-e66b-4f0a-9639-aac0feba5432=VmDevice {vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, deviceId=6ad955b3-e66b-4f0a-9639-aac0feba5432, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}}, device_2528f132-6ebf-4acb-8b86-aecf0bc31cc3=VmDevice {vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, deviceId=2528f132-6ebf-4acb-8b86-aecf0bc31cc3, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}}, device_2528f132-6ebf-4acb-8b86-aecf0bc31cc3device_ffc79dbb-122a-4ad9-a297-f0857192f939=VmDevice {vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, deviceId=ffc79dbb-122a-4ad9-a297-f0857192f939, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}}} >2014-01-09 12:52:17,448 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-47) FINISH, CreateVDSCommand, log id: 7dcba62c >2014-01-09 12:52:17,451 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-6-thread-47) FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 226d52fc >2014-01-09 12:52:17,451 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-6-thread-47) Lock freed to object EngineLock [exclusiveLocks= key: be5348a1-85c7-48dd-a8ad-8a0464643bd8 value: VM >, sharedLocks= ] >2014-01-09 12:52:17,453 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-47) Correlation ID: 676c3913, Job ID: 731719a1-4ffc-4610-a7dc-4996a075d1c6, Call Stack: null, Custom Event ID: -1, Message: VM Apache01 was started by ryan-a (Host: ovirt01). >2014-01-09 12:52:19,799 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-48) VM Apache01 be5348a1-85c7-48dd-a8ad-8a0464643bd8 moved from WaitForLaunch --> PoweringUp >2014-01-09 12:52:19,803 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-48) START, FullListVdsCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01], vmIds=[be5348a1-85c7-48dd-a8ad-8a0464643bd8]), log id: 35ff9b9 >2014-01-09 12:52:19,813 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-48) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@3d05a663, log id: 35ff9b9 >2014-01-09 12:52:40,983 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp--127.0.0.1-8702-4) [2819cdef] Lock Acquired to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 12:52:41,007 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-50) [2819cdef] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 Type: VM >2014-01-09 12:52:41,022 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-50) [2819cdef] START, MigrateVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=14f2245f-b5a9-48b0-aa78-c8cab04d9be7, srcHost=192.168.100.22, dstVdsId=f2d7a8a9-e77e-4e9a-8031-23a0731407c8, dstHost=192.168.100.21:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 639b1d6 >2014-01-09 12:52:41,023 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-50) [2819cdef] VdsBroker::migrate::Entered (vm_guid=14f2245f-b5a9-48b0-aa78-c8cab04d9be7, srcHost=192.168.100.22, dstHost=192.168.100.21:54321, method=online >2014-01-09 12:52:41,025 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-50) [2819cdef] START, MigrateBrokerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=14f2245f-b5a9-48b0-aa78-c8cab04d9be7, srcHost=192.168.100.22, dstVdsId=f2d7a8a9-e77e-4e9a-8031-23a0731407c8, dstHost=192.168.100.21:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 69b9fe93 >2014-01-09 12:52:41,035 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-50) [2819cdef] FINISH, MigrateBrokerVDSCommand, log id: 69b9fe93 >2014-01-09 12:52:41,039 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-50) [2819cdef] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 639b1d6 >2014-01-09 12:52:41,044 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-50) [2819cdef] Correlation ID: 2819cdef, Job ID: 9ff4ae76-841f-4e28-8390-7cb51ecb687c, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: syslog00, Source: ovirt02, Destination: ovirt01, User: ryan-a). >2014-01-09 12:52:41,521 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-32) START, FullListVdsCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02], vmIds=[14f2245f-b5a9-48b0-aa78-c8cab04d9be7]), log id: 70a1c1a2 >2014-01-09 12:52:41,532 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-32) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@3a62b803, log id: 70a1c1a2 >2014-01-09 12:52:41,535 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-32) Received a memballoon Device without an address when processing VM 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 devices, skipping device: {alias=balloon0, specParams={model=none}, target=3145728, device=memballoon, type=balloon} >2014-01-09 12:52:44,064 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:52:47,092 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-11) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:52:50,126 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-24) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:52:53,162 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-45) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:52:56,193 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:52:59,221 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-34) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:53:02,250 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-51) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:53:05,285 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-61) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:53:08,322 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-21) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:53:11,353 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:53:14,381 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:53:17,411 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:53:20,445 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) VM Apache01 be5348a1-85c7-48dd-a8ad-8a0464643bd8 moved from PoweringUp --> Up >2014-01-09 12:53:20,449 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:53:20,460 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-77) Correlation ID: 676c3913, Job ID: 731719a1-4ffc-4610-a7dc-4996a075d1c6, Call Stack: null, Custom Event ID: -1, Message: VM Apache01 started on Host ovirt01 >2014-01-09 12:53:20,465 INFO [org.ovirt.engine.core.vdsbroker.FailedToRunVmVDSCommand] (pool-6-thread-50) START, FailedToRunVmVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5), log id: 6d094920 >2014-01-09 12:53:20,467 INFO [org.ovirt.engine.core.vdsbroker.FailedToRunVmVDSCommand] (pool-6-thread-50) FINISH, FailedToRunVmVDSCommand, log id: 6d094920 >2014-01-09 12:53:23,482 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:53:26,502 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:53:29,520 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-95) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:53:32,540 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:53:35,576 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-79) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:53:38,603 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:53:41,624 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:53:44,647 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-19) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:53:47,668 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:53:50,702 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-57) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:53:53,729 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-32) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:53:56,749 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-53) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:53:59,769 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-49) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:02,793 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-24) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:05,828 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:08,855 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:09,554 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp--127.0.0.1-8702-3) [3ff3d884] Lock Acquired to object EngineLock [exclusiveLocks= key: 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa value: VM >, sharedLocks= ] >2014-01-09 12:54:09,579 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-48) [3ff3d884] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa Type: VM >2014-01-09 12:54:09,595 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-48) [3ff3d884] START, MigrateVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, srcHost=192.168.100.22, dstVdsId=f2d7a8a9-e77e-4e9a-8031-23a0731407c8, dstHost=192.168.100.21:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 6dc19be9 >2014-01-09 12:54:09,597 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [3ff3d884] VdsBroker::migrate::Entered (vm_guid=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, srcHost=192.168.100.22, dstHost=192.168.100.21:54321, method=online >2014-01-09 12:54:09,599 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [3ff3d884] START, MigrateBrokerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, srcHost=192.168.100.22, dstVdsId=f2d7a8a9-e77e-4e9a-8031-23a0731407c8, dstHost=192.168.100.21:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 598472cf >2014-01-09 12:54:09,613 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [3ff3d884] FINISH, MigrateBrokerVDSCommand, log id: 598472cf >2014-01-09 12:54:09,628 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-48) [3ff3d884] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 6dc19be9 >2014-01-09 12:54:09,632 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) [3ff3d884] Correlation ID: 3ff3d884, Job ID: 62267aae-f00a-44fd-9bdd-34ef403b4314, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: VETS, Source: ovirt02, Destination: ovirt01, User: ryan-a). >2014-01-09 12:54:11,937 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) RefreshVmList vm id 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:11,939 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:14,966 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-51) RefreshVmList vm id 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:14,971 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-51) START, FullListVdsCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01], vmIds=[14f2245f-b5a9-48b0-aa78-c8cab04d9be7]), log id: 44e35cc8 >2014-01-09 12:54:14,981 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-51) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@754f5fb8, log id: 44e35cc8 >2014-01-09 12:54:14,983 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-51) Received a memballoon Device without an address when processing VM 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 devices, skipping device: {alias=balloon0, specParams={model=none}, target=3145728, device=memballoon, type=balloon} >2014-01-09 12:54:15,005 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-51) Correlation ID: 2819cdef, Job ID: 9ff4ae76-841f-4e28-8390-7cb51ecb687c, Call Stack: null, Custom Event ID: -1, Message: Migration completed (VM: syslog00, Source: ovirt02, Destination: ovirt01, Duration: 33 sec). >2014-01-09 12:54:15,008 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (DefaultQuartzScheduler_Worker-51) Lock freed to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 12:54:15,606 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-27) START, DestroyVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=14f2245f-b5a9-48b0-aa78-c8cab04d9be7, force=false, secondsToWait=0, gracefully=false), log id: 75181d8b >2014-01-09 12:54:16,882 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-27) FINISH, DestroyVDSCommand, log id: 75181d8b >2014-01-09 12:54:16,885 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-27) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 status = Down on vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 12:54:18,029 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-43) RefreshVmList vm id 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:21,074 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) RefreshVmList vm id 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:24,148 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-41) START, FullListVdsCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01], vmIds=[60ace091-8a4e-4047-9ac0-c0ea95e3c5fa]), log id: 1ad43340 >2014-01-09 12:54:24,157 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-41) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@30682cc8, log id: 1ad43340 >2014-01-09 12:54:24,160 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-41) Received a memballoon Device without an address when processing VM 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa devices, skipping device: {alias=balloon0, specParams={model=none}, target=393216, device=memballoon, type=balloon} >2014-01-09 12:54:24,173 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-41) Correlation ID: 3ff3d884, Job ID: 62267aae-f00a-44fd-9bdd-34ef403b4314, Call Stack: null, Custom Event ID: -1, Message: Migration completed (VM: VETS, Source: ovirt02, Destination: ovirt01, Duration: 14 sec). >2014-01-09 12:54:24,177 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (DefaultQuartzScheduler_Worker-41) Lock freed to object EngineLock [exclusiveLocks= key: 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa value: VM >, sharedLocks= ] >2014-01-09 12:54:25,936 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-50) START, DestroyVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, force=false, secondsToWait=0, gracefully=false), log id: 5fd5b928 >2014-01-09 12:54:26,456 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-50) FINISH, DestroyVDSCommand, log id: 5fd5b928 >2014-01-09 12:54:26,458 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) RefreshVmList vm id 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa status = Down on vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 12:54:29,171 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp--127.0.0.1-8702-9) [492a9801] Lock Acquired to object EngineLock [exclusiveLocks= key: fc077d22-f9a1-45e4-839d-840c128a10c6 value: VM >, sharedLocks= ] >2014-01-09 12:54:29,223 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-48) [492a9801] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: fc077d22-f9a1-45e4-839d-840c128a10c6 Type: VM >2014-01-09 12:54:29,234 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-48) [492a9801] START, MigrateVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=fc077d22-f9a1-45e4-839d-840c128a10c6, srcHost=192.168.100.22, dstVdsId=f2d7a8a9-e77e-4e9a-8031-23a0731407c8, dstHost=192.168.100.21:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 49634f61 >2014-01-09 12:54:29,235 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [492a9801] VdsBroker::migrate::Entered (vm_guid=fc077d22-f9a1-45e4-839d-840c128a10c6, srcHost=192.168.100.22, dstHost=192.168.100.21:54321, method=online >2014-01-09 12:54:29,236 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [492a9801] START, MigrateBrokerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=fc077d22-f9a1-45e4-839d-840c128a10c6, srcHost=192.168.100.22, dstVdsId=f2d7a8a9-e77e-4e9a-8031-23a0731407c8, dstHost=192.168.100.21:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 6d415f49 >2014-01-09 12:54:29,245 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [492a9801] FINISH, MigrateBrokerVDSCommand, log id: 6d415f49 >2014-01-09 12:54:29,249 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-48) [492a9801] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 49634f61 >2014-01-09 12:54:29,259 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) [492a9801] Correlation ID: 492a9801, Job ID: 01941b23-a1cc-44eb-a58a-a6721a2c365f, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: Finance, Source: ovirt02, Destination: ovirt01, User: ryan-a). >2014-01-09 12:54:30,208 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:33,230 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:36,264 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:36,536 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp--127.0.0.1-8702-9) [3383de81] Lock Acquired to object EngineLock [exclusiveLocks= key: 872c3dea-6984-48fa-99c3-354fd367c3f9 value: VM >, sharedLocks= ] >2014-01-09 12:54:36,561 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-48) [3383de81] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: 872c3dea-6984-48fa-99c3-354fd367c3f9 Type: VM >2014-01-09 12:54:36,577 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-48) [3383de81] START, MigrateVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, srcHost=192.168.100.22, dstVdsId=f2d7a8a9-e77e-4e9a-8031-23a0731407c8, dstHost=192.168.100.21:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 871a3ef >2014-01-09 12:54:36,578 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [3383de81] VdsBroker::migrate::Entered (vm_guid=872c3dea-6984-48fa-99c3-354fd367c3f9, srcHost=192.168.100.22, dstHost=192.168.100.21:54321, method=online >2014-01-09 12:54:36,580 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [3383de81] START, MigrateBrokerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, srcHost=192.168.100.22, dstVdsId=f2d7a8a9-e77e-4e9a-8031-23a0731407c8, dstHost=192.168.100.21:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 76be832d >2014-01-09 12:54:36,590 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [3383de81] FINISH, MigrateBrokerVDSCommand, log id: 76be832d >2014-01-09 12:54:36,598 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-48) [3383de81] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 871a3ef >2014-01-09 12:54:36,603 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) [3383de81] Correlation ID: 3383de81, Job ID: 73a54fb1-e956-4e19-84ed-b8b4cad020e1, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: External_Email, Source: ovirt02, Destination: ovirt01, User: ryan-a). >2014-01-09 12:54:39,311 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:39,312 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:42,338 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-95) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:42,340 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-95) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:45,365 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-48) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:45,367 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-48) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:46,903 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp--127.0.0.1-8702-10) [4db59c02] Lock Acquired to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 12:54:46,953 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-48) [4db59c02] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: 33f88d50-30eb-45c7-9850-733da17ed7a4 Type: VM >2014-01-09 12:54:46,968 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-48) [4db59c02] START, MigrateVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, srcHost=192.168.100.22, dstVdsId=f2d7a8a9-e77e-4e9a-8031-23a0731407c8, dstHost=192.168.100.21:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 1acd50c1 >2014-01-09 12:54:46,969 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [4db59c02] VdsBroker::migrate::Entered (vm_guid=33f88d50-30eb-45c7-9850-733da17ed7a4, srcHost=192.168.100.22, dstHost=192.168.100.21:54321, method=online >2014-01-09 12:54:46,971 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [4db59c02] START, MigrateBrokerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, srcHost=192.168.100.22, dstVdsId=f2d7a8a9-e77e-4e9a-8031-23a0731407c8, dstHost=192.168.100.21:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 64383d21 >2014-01-09 12:54:46,997 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [4db59c02] FINISH, MigrateBrokerVDSCommand, log id: 64383d21 >2014-01-09 12:54:47,101 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-48) [4db59c02] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 1acd50c1 >2014-01-09 12:54:47,105 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) [4db59c02] Correlation ID: 4db59c02, Job ID: 70d60f91-0d6e-432d-896f-9631b513a5a7, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: DC-02, Source: ovirt02, Destination: ovirt01, User: ryan-a). >2014-01-09 12:54:48,400 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-83) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:48,401 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-83) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:48,402 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-83) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:51,442 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:51,444 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:51,477 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:54,516 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-59) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:54,517 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-59) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:54,518 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-59) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:57,550 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:57,552 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:54:57,553 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:00,591 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:00,592 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:00,594 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:03,626 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:03,627 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:03,628 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:06,868 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:06,869 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:06,902 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:09,941 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:09,942 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:09,944 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:12,976 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-55) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:12,977 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-55) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:12,979 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-55) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:16,012 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:16,013 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:16,015 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:19,047 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-16) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:19,048 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-16) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:19,049 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-16) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:22,086 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-37) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:22,087 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-37) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:22,099 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-37) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:25,142 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:25,143 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:25,145 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:28,177 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-21) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:28,179 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-21) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:28,180 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-21) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:30,554 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-18) START, DestroyVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=fc077d22-f9a1-45e4-839d-840c128a10c6, force=false, secondsToWait=0, gracefully=false), log id: 54fbf6ea >2014-01-09 12:55:31,073 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-18) FINISH, DestroyVDSCommand, log id: 54fbf6ea >2014-01-09 12:55:31,085 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Handing over VM Finance fc077d22-f9a1-45e4-839d-840c128a10c6 to Host f2d7a8a9-e77e-4e9a-8031-23a0731407c8. Setting VM to status MigratingTo >2014-01-09 12:55:31,086 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) vm Finance running in db and not running in vds - add to rerun treatment. vds ovirt02 >2014-01-09 12:55:31,212 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-56) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:31,213 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-56) VM Finance fc077d22-f9a1-45e4-839d-840c128a10c6 moved from MigratingTo --> Up >2014-01-09 12:55:31,214 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-56) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:31,215 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-56) START, FullListVdsCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01], vmIds=[fc077d22-f9a1-45e4-839d-840c128a10c6]), log id: 3bebc8bc >2014-01-09 12:55:31,224 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-56) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@697c53de, log id: 3bebc8bc >2014-01-09 12:55:31,227 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-56) Received a memballoon Device without an address when processing VM fc077d22-f9a1-45e4-839d-840c128a10c6 devices, skipping device: {alias=balloon0, specParams={model=none}, target=2097152, device=memballoon, type=balloon} >2014-01-09 12:55:31,239 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-56) Correlation ID: 492a9801, Job ID: 01941b23-a1cc-44eb-a58a-a6721a2c365f, Call Stack: null, Custom Event ID: -1, Message: Migration completed (VM: Finance, Source: ovirt02, Destination: ovirt01, Duration: 2 sec). >2014-01-09 12:55:31,242 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (DefaultQuartzScheduler_Worker-56) Lock freed to object EngineLock [exclusiveLocks= key: fc077d22-f9a1-45e4-839d-840c128a10c6 value: VM >, sharedLocks= ] >2014-01-09 12:55:34,266 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-10) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:34,268 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-10) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:37,312 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-64) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:37,327 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-64) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:40,362 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:40,364 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:43,216 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-52) START, DestroyVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, force=false, secondsToWait=0, gracefully=false), log id: 1a8f844c >2014-01-09 12:55:43,397 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:43,401 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-82) START, FullListVdsCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01], vmIds=[872c3dea-6984-48fa-99c3-354fd367c3f9]), log id: 5c7965a9 >2014-01-09 12:55:43,412 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-82) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@26c0f059, log id: 5c7965a9 >2014-01-09 12:55:43,423 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-82) Correlation ID: 3383de81, Job ID: 73a54fb1-e956-4e19-84ed-b8b4cad020e1, Call Stack: null, Custom Event ID: -1, Message: Migration completed (VM: External_Email, Source: ovirt02, Destination: ovirt01, Duration: 6 sec). >2014-01-09 12:55:43,426 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (DefaultQuartzScheduler_Worker-82) Lock freed to object EngineLock [exclusiveLocks= key: 872c3dea-6984-48fa-99c3-354fd367c3f9 value: VM >, sharedLocks= ] >2014-01-09 12:55:44,573 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-52) FINISH, DestroyVDSCommand, log id: 1a8f844c >2014-01-09 12:55:44,575 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Handing over VM External_Email 872c3dea-6984-48fa-99c3-354fd367c3f9 to Host f2d7a8a9-e77e-4e9a-8031-23a0731407c8. Setting VM to status MigratingTo >2014-01-09 12:55:44,576 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) vm External_Email running in db and not running in vds - add to rerun treatment. vds ovirt02 >2014-01-09 12:55:46,455 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:55:46,456 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) VM External_Email 872c3dea-6984-48fa-99c3-354fd367c3f9 moved from MigratingTo --> Up >2014-01-09 12:55:46,457 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-75) START, FullListVdsCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01], vmIds=[872c3dea-6984-48fa-99c3-354fd367c3f9]), log id: 7ff85e5c >2014-01-09 12:55:46,467 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-75) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@6feee692, log id: 7ff85e5c >2014-01-09 12:55:49,500 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-28) START, FullListVdsCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01], vmIds=[33f88d50-30eb-45c7-9850-733da17ed7a4]), log id: 294e9d93 >2014-01-09 12:55:49,509 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-28) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@21069b8a, log id: 294e9d93 >2014-01-09 12:55:49,520 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-28) Correlation ID: 4db59c02, Job ID: 70d60f91-0d6e-432d-896f-9631b513a5a7, Call Stack: null, Custom Event ID: -1, Message: Migration completed (VM: DC-02, Source: ovirt02, Destination: ovirt01, Duration: 2 sec). >2014-01-09 12:55:49,523 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (DefaultQuartzScheduler_Worker-28) Lock freed to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 12:55:50,664 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-87) START, DestroyVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, force=false, secondsToWait=0, gracefully=false), log id: 7254e1cf >2014-01-09 12:55:53,017 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-87) FINISH, DestroyVDSCommand, log id: 7254e1cf >2014-01-09 12:55:53,019 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-87) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 status = Down on vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 12:56:27,053 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp--127.0.0.1-8702-11) [191011a1] Lock Acquired to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 12:56:27,106 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-48) [191011a1] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: b6972001-2bd6-4cea-9c15-c046674754d4 Type: VM >2014-01-09 12:56:27,116 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-48) [191011a1] START, MigrateVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=b6972001-2bd6-4cea-9c15-c046674754d4, srcHost=192.168.100.22, dstVdsId=f2d7a8a9-e77e-4e9a-8031-23a0731407c8, dstHost=192.168.100.21:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 21621f09 >2014-01-09 12:56:27,118 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [191011a1] VdsBroker::migrate::Entered (vm_guid=b6972001-2bd6-4cea-9c15-c046674754d4, srcHost=192.168.100.22, dstHost=192.168.100.21:54321, method=online >2014-01-09 12:56:27,119 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [191011a1] START, MigrateBrokerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=b6972001-2bd6-4cea-9c15-c046674754d4, srcHost=192.168.100.22, dstVdsId=f2d7a8a9-e77e-4e9a-8031-23a0731407c8, dstHost=192.168.100.21:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 1e3a3176 >2014-01-09 12:56:27,130 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [191011a1] FINISH, MigrateBrokerVDSCommand, log id: 1e3a3176 >2014-01-09 12:56:27,134 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-48) [191011a1] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 21621f09 >2014-01-09 12:56:27,138 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) [191011a1] Correlation ID: 191011a1, Job ID: 3f4cd3a7-fa11-493b-989b-8921256a6545, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: DC-01, Source: ovirt02, Destination: ovirt01, User: ryan-a). >2014-01-09 12:56:28,890 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:56:31,912 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-40) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:56:34,935 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:56:38,009 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-15) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:56:41,050 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-61) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:56:44,072 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-56) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:56:47,093 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-41) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:56:50,115 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:56:53,170 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:56:56,206 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:56:59,229 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:57:02,249 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:57:05,270 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:57:08,331 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-81) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:57:11,367 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-79) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:57:14,389 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:57:17,410 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-68) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 12:57:25,444 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-19) START, FullListVdsCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01], vmIds=[b6972001-2bd6-4cea-9c15-c046674754d4]), log id: 1595cc64 >2014-01-09 12:57:25,454 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-19) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@60c2ec12, log id: 1595cc64 >2014-01-09 12:57:25,466 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-19) Correlation ID: 191011a1, Job ID: 3f4cd3a7-fa11-493b-989b-8921256a6545, Call Stack: null, Custom Event ID: -1, Message: Migration completed (VM: DC-01, Source: ovirt02, Destination: ovirt01, Duration: 58 sec). >2014-01-09 12:57:25,468 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (DefaultQuartzScheduler_Worker-19) Lock freed to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 12:57:25,566 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-22) START, DestroyVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=b6972001-2bd6-4cea-9c15-c046674754d4, force=false, secondsToWait=0, gracefully=false), log id: 2c154c9f >2014-01-09 12:57:26,066 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-22) FINISH, DestroyVDSCommand, log id: 2c154c9f >2014-01-09 12:57:26,069 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) Handing over VM DC-01 b6972001-2bd6-4cea-9c15-c046674754d4 to Host f2d7a8a9-e77e-4e9a-8031-23a0731407c8. Setting VM to status MigratingTo >2014-01-09 12:57:26,070 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) vm DC-01 running in db and not running in vds - add to rerun treatment. vds ovirt02 >2014-01-09 12:57:28,526 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) VM DC-01 b6972001-2bd6-4cea-9c15-c046674754d4 moved from MigratingTo --> Up >2014-01-09 12:57:28,533 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-85) START, FullListVdsCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01], vmIds=[b6972001-2bd6-4cea-9c15-c046674754d4]), log id: 5df84f38 >2014-01-09 12:57:28,544 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-85) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@7de19408, log id: 5df84f38 >2014-01-09 12:58:16,126 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (ajp--127.0.0.1-8702-1) [3844cfdc] Lock Acquired to object EngineLock [exclusiveLocks= , sharedLocks= key: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d value: POOL >] >2014-01-09 12:58:16,131 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (pool-6-thread-50) [3844cfdc] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 12:58:16,134 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-6-thread-50) [3844cfdc] START, SetVdsStatusVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, status=PreparingForMaintenance, nonOperationalReason=NONE), log id: 3378e01d >2014-01-09 12:58:16,138 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-6-thread-50) [3844cfdc] VDS ovirt02 is spm and moved from up calling ResetIrs. >2014-01-09 12:58:16,139 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (pool-6-thread-50) [3844cfdc] START, ResetIrsVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, vdsId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, ignoreStopFailed = false), log id: 3811a526 >2014-01-09 12:58:16,141 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-6-thread-50) [3844cfdc] START, SpmStopVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 382d59 >2014-01-09 12:58:16,153 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-6-thread-50) [3844cfdc] SpmStopVDSCommand::Stopping SPM on vds ovirt02, pool id 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 12:58:20,087 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-6-thread-50) [3844cfdc] FINISH, SpmStopVDSCommand, log id: 382d59 >2014-01-09 12:58:20,094 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (pool-6-thread-50) [3844cfdc] FINISH, ResetIrsVDSCommand, log id: 3811a526 >2014-01-09 12:58:20,094 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-60) IrsBroker::Failed::GetStoragePoolInfoVDS due to: SocketException: Socket closed >2014-01-09 12:58:20,095 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-6-thread-50) [3844cfdc] FINISH, SetVdsStatusVDSCommand, log id: 3378e01d >2014-01-09 12:58:20,097 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (pool-6-thread-50) [3844cfdc] Lock freed to object EngineLock [exclusiveLocks= , sharedLocks= key: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d value: POOL >] >2014-01-09 12:58:20,103 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-60) IRS failover failed - cant allocate vds server >2014-01-09 12:58:20,109 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (pool-6-thread-50) [3844cfdc] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 12:58:20,116 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-50) [3844cfdc] Correlation ID: 3844cfdc, Job ID: ee8374bb-6e54-4e46-8e76-5aa9e52bae18, Call Stack: null, Custom Event ID: -1, Message: Host ovirt02 was switched to Maintenance mode by ryan-a. >2014-01-09 12:58:20,129 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 : ovirt02 >2014-01-09 12:58:20,134 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-6-thread-48) Clearing cache of pool: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d for problematic entities of VDS: ovirt02. >2014-01-09 12:58:20,137 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-47) START, DisconnectStoragePoolVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, vds_spm_id = 2), log id: bc5762b >2014-01-09 12:58:24,457 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-47) FINISH, DisconnectStoragePoolVDSCommand, log id: bc5762b >2014-01-09 12:58:24,458 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-47) Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 12:58:25,268 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-47) START, DisconnectStorageServerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = ISCSI, connectionList = [{ id: 623aa506-12d1-4437-8e81-f39b2317f3dd, connection: 192.168.101.101, iqn: iqn.2013-11.local.cytechservices.n16000pro:iscsi.hosts-tgt00.raid0, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 59641e6 >2014-01-09 12:58:28,480 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-47) FINISH, DisconnectStorageServerVDSCommand, return: {623aa506-12d1-4437-8e81-f39b2317f3dd=0}, log id: 59641e6 >2014-01-09 12:58:28,481 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-47) START, DisconnectStorageServerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 210997b7-9098-48d6-8e97-9689085aa4b0, connection: 192.168.101.101:/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: AUTO, nfsRetrans: null, nfsTimeo: null };]), log id: 3413fead >2014-01-09 12:58:30,107 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-76) Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 12:58:30,112 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-76) Correlation ID: 548b0416, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center CSEnt. Setting status to Non Responsive. >2014-01-09 12:58:30,125 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-76) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 12:58:30,130 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-76) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 12:58:30,132 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-76) SPM selection - vds seems as spm ovirt02 >2014-01-09 12:58:30,134 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-76) START, SpmStopVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 23bbd24f >2014-01-09 12:58:30,145 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-76) Command org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand return value > >TaskStatusListReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=654, mMessage=Not SPM: ()]] > >2014-01-09 12:58:30,147 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-76) HostName = ovirt02 >2014-01-09 12:58:30,148 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-76) Command HSMGetAllTasksStatusesVDS execution failed. Exception: IRSNonOperationalException: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM: () >2014-01-09 12:58:30,149 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-76) FINISH, SpmStopVDSCommand, log id: 23bbd24f >2014-01-09 12:58:30,149 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-76) spm stop succeeded, continuing with spm selection >2014-01-09 12:58:30,151 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-76) starting spm on vds ovirt01, storage pool CSEnt, prevId 2, LVER 10 >2014-01-09 12:58:30,152 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-76) START, SpmStartVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, prevId=2, prevLVER=10, storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=true), log id: 1907a0bd >2014-01-09 12:58:30,165 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-76) spmStart polling started: taskId = 916c911d-e61c-47bb-a5bc-630fba9e674b >2014-01-09 12:58:32,647 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-47) FINISH, DisconnectStorageServerVDSCommand, return: {210997b7-9098-48d6-8e97-9689085aa4b0=0}, log id: 3413fead >2014-01-09 12:58:32,649 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-47) START, DisconnectStorageServerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 4e5d4030-1bd2-4487-b08f-2135eba3f11f, connection: 192.168.101.101:/vms, iqn: null, vfsType: null, mountOptions: null, nfsVersion: V4, nfsRetrans: null, nfsTimeo: null };]), log id: 6d61d05a >2014-01-09 12:58:34,245 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-76) spmStart polling ended: taskId = 916c911d-e61c-47bb-a5bc-630fba9e674b task status = finished >2014-01-09 12:58:34,253 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-76) spmStart polling ended, spm status: SPM >2014-01-09 12:58:34,255 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-76) START, HSMClearTaskVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, taskId=916c911d-e61c-47bb-a5bc-630fba9e674b), log id: f570265 >2014-01-09 12:58:34,264 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-76) FINISH, HSMClearTaskVDSCommand, log id: f570265 >2014-01-09 12:58:34,264 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-76) FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.businessentities.SpmStatusResult@4e6698dc, log id: 1907a0bd >2014-01-09 12:58:34,266 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-76) Initialize Irs proxy from vds: 192.168.100.21 >2014-01-09 12:58:34,270 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-76) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host ovirt01 (Address: 192.168.100.21). >2014-01-09 12:58:34,280 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-6-thread-50) START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false), log id: 2a867479 >2014-01-09 12:58:34,301 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-6-thread-50) -- executeIrsBrokerCommand: Attempting on storage pool 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 12:58:34,303 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (pool-6-thread-50) START, HSMGetAllTasksInfoVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8), log id: a33580 >2014-01-09 12:58:34,313 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (pool-6-thread-50) FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: a33580 >2014-01-09 12:58:34,313 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-6-thread-50) FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 2a867479 >2014-01-09 12:58:34,314 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (pool-6-thread-50) Discovered no tasks on Storage Pool CSEnt >2014-01-09 12:58:35,774 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-47) FINISH, DisconnectStorageServerVDSCommand, return: {4e5d4030-1bd2-4487-b08f-2135eba3f11f=0}, log id: 6d61d05a >2014-01-09 13:13:30,431 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (ajp--127.0.0.1-8702-5) [5a0d604] Lock Acquired to object EngineLock [exclusiveLocks= key: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 value: VDS_FENCE >, sharedLocks= ] >2014-01-09 13:13:30,433 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (pool-6-thread-47) [5a0d604] Running command: FenceVdsManualyCommand internal: false. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 13:13:30,437 INFO [org.ovirt.engine.core.bll.ClearNonResponsiveVdsVmsCommand] (pool-6-thread-47) Running command: ClearNonResponsiveVdsVmsCommand internal: true. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 13:13:30,440 INFO [org.ovirt.engine.core.vdsbroker.UpdateVdsVMsClearedVDSCommand] (pool-6-thread-47) START, UpdateVdsVMsClearedVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5), log id: 60977e36 >2014-01-09 13:13:30,441 INFO [org.ovirt.engine.core.vdsbroker.UpdateVdsVMsClearedVDSCommand] (pool-6-thread-47) FINISH, UpdateVdsVMsClearedVDSCommand, log id: 60977e36 >2014-01-09 13:13:30,445 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-47) Correlation ID: 35989b13, Call Stack: null, Custom Event ID: -1, Message: All VMs' status on Non Responsive Host ovirt02 were changed to 'Down' by ryan-a >2014-01-09 13:13:30,452 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-47) Correlation ID: 5a0d604, Call Stack: null, Custom Event ID: -1, Message: Manual fence for host ovirt02 was started. >2014-01-09 13:13:30,453 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (pool-6-thread-47) Lock freed to object EngineLock [exclusiveLocks= key: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 value: VDS_FENCE >, sharedLocks= ] >2014-01-09 13:13:34,560 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp--127.0.0.1-8702-4) [39640d2c] Lock Acquired to object EngineLock [exclusiveLocks= key: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 value: VDS >, sharedLocks= ] >2014-01-09 13:13:34,571 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-6-thread-47) [39640d2c] Running command: ActivateVdsCommand internal: false. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 13:13:34,572 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-6-thread-47) [39640d2c] Before acquiring lock in order to prevent monitoring for host ovirt02 from data-center CSEnt >2014-01-09 13:13:34,573 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-6-thread-47) [39640d2c] Lock acquired, from now a monitoring of host will be skipped for host ovirt02 from data-center CSEnt >2014-01-09 13:13:34,578 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-6-thread-47) [39640d2c] START, SetVdsStatusVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, status=Unassigned, nonOperationalReason=NONE), log id: 3a658a6d >2014-01-09 13:13:34,580 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-6-thread-47) [39640d2c] FINISH, SetVdsStatusVDSCommand, log id: 3a658a6d >2014-01-09 13:13:34,582 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (pool-6-thread-47) [39640d2c] START, ActivateVdsVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5), log id: 361d918a >2014-01-09 13:13:34,942 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (pool-6-thread-47) [39640d2c] START, GetHardwareInfoVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02]), log id: 6c72e809 >2014-01-09 13:13:35,116 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (pool-6-thread-47) [39640d2c] FINISH, GetHardwareInfoVDSCommand, log id: 6c72e809 >2014-01-09 13:13:35,127 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (pool-6-thread-47) Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 13:13:35,129 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (pool-6-thread-47) Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 13:13:35,130 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (pool-6-thread-47) FINISH, ActivateVdsVDSCommand, return: Host[ovirt02], log id: 361d918a >2014-01-09 13:13:35,232 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-6-thread-47) Activate finished. Lock released. Monitoring can run now for host ovirt02 from data-center CSEnt >2014-01-09 13:13:35,235 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-47) Correlation ID: 39640d2c, Job ID: bb1d3361-13f9-47ed-8a9e-9fa054858c8c, Call Stack: null, Custom Event ID: -1, Message: Host ovirt02 was activated by ryan-a. >2014-01-09 13:13:35,237 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-6-thread-47) Lock freed to object EngineLock [exclusiveLocks= key: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 value: VDS >, sharedLocks= ] >2014-01-09 13:13:36,382 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-61) START, GetHardwareInfoVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02]), log id: 75fe514f >2014-01-09 13:13:36,391 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-61) FINISH, GetHardwareInfoVDSCommand, log id: 75fe514f >2014-01-09 13:13:36,402 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-61) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host ovirt02. >2014-01-09 13:13:36,410 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-61) Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 13:13:36,412 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-61) Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 13:13:37,224 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-61) START, ConnectStorageServerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = ISCSI, connectionList = [{ id: 623aa506-12d1-4437-8e81-f39b2317f3dd, connection: 192.168.101.101, iqn: iqn.2013-11.local.cytechservices.n16000pro:iscsi.hosts-tgt00.raid0, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 4629838d >2014-01-09 13:13:40,200 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-61) FINISH, ConnectStorageServerVDSCommand, return: {623aa506-12d1-4437-8e81-f39b2317f3dd=0}, log id: 4629838d >2014-01-09 13:13:40,201 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-61) Host ovirt02 storage connection was succeeded >2014-01-09 13:13:40,203 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-61) START, ConnectStorageServerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 210997b7-9098-48d6-8e97-9689085aa4b0, connection: 192.168.101.101:/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: AUTO, nfsRetrans: null, nfsTimeo: null };]), log id: 38764493 >2014-01-09 13:13:40,556 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-61) FINISH, ConnectStorageServerVDSCommand, return: {210997b7-9098-48d6-8e97-9689085aa4b0=0}, log id: 38764493 >2014-01-09 13:13:40,557 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-61) Host ovirt02 storage connection was succeeded >2014-01-09 13:13:40,559 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-61) START, ConnectStorageServerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 4e5d4030-1bd2-4487-b08f-2135eba3f11f, connection: 192.168.101.101:/vms, iqn: null, vfsType: null, mountOptions: null, nfsVersion: V4, nfsRetrans: null, nfsTimeo: null };]), log id: 3c37fbfe >2014-01-09 13:13:40,624 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-61) FINISH, ConnectStorageServerVDSCommand, return: {4e5d4030-1bd2-4487-b08f-2135eba3f11f=0}, log id: 3c37fbfe >2014-01-09 13:13:40,625 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-61) Host ovirt02 storage connection was succeeded >2014-01-09 13:13:40,629 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-6-thread-50) START, ConnectStoragePoolVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, vds_spm_id = 2, masterDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, masterVersion = 2), log id: 115db484 >2014-01-09 13:13:47,019 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-6-thread-50) FINISH, ConnectStoragePoolVDSCommand, log id: 115db484 >2014-01-09 13:13:47,035 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-61) START, SetMOMPolicyParametersVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5), log id: 784a225b >2014-01-09 13:13:47,113 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-61) FINISH, SetMOMPolicyParametersVDSCommand, log id: 784a225b >2014-01-09 13:13:47,116 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-61) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host ovirt02. >2014-01-09 13:13:47,121 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-61) Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 13:13:47,124 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-61) Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 13:14:29,495 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (pool-6-thread-47) [6a43df86] Start running CanDoAction for command number 1/4 (Command type: MigrateVmToServer) >2014-01-09 13:14:29,496 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (pool-6-thread-49) [6cbe4c4] Start running CanDoAction for command number 4/4 (Command type: MigrateVmToServer) >2014-01-09 13:14:29,495 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (pool-6-thread-50) [e080ceb] Start running CanDoAction for command number 2/4 (Command type: MigrateVmToServer) >2014-01-09 13:14:29,495 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (pool-6-thread-48) [32881f7f] Start running CanDoAction for command number 3/4 (Command type: MigrateVmToServer) >2014-01-09 13:14:29,500 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-47) [6a43df86] Lock Acquired to object EngineLock [exclusiveLocks= key: be5348a1-85c7-48dd-a8ad-8a0464643bd8 value: VM >, sharedLocks= ] >2014-01-09 13:14:29,501 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-49) [6cbe4c4] Lock Acquired to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 13:14:29,503 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-50) [e080ceb] Lock Acquired to object EngineLock [exclusiveLocks= key: fc077d22-f9a1-45e4-839d-840c128a10c6 value: VM >, sharedLocks= ] >2014-01-09 13:14:29,514 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-48) [32881f7f] Lock Acquired to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 13:14:29,520 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (pool-6-thread-47) [6a43df86] Finish handling CanDoAction for command number 1/4 (Command type: MigrateVmToServer) >2014-01-09 13:14:29,525 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (pool-6-thread-50) [e080ceb] Finish handling CanDoAction for command number 2/4 (Command type: MigrateVmToServer) >2014-01-09 13:14:29,547 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (pool-6-thread-49) [6cbe4c4] Finish handling CanDoAction for command number 4/4 (Command type: MigrateVmToServer) >2014-01-09 13:14:29,548 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (pool-6-thread-48) [32881f7f] Finish handling CanDoAction for command number 3/4 (Command type: MigrateVmToServer) >2014-01-09 13:14:29,556 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-48) [6a43df86] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: be5348a1-85c7-48dd-a8ad-8a0464643bd8 Type: VM >2014-01-09 13:14:29,566 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-48) [6a43df86] START, MigrateVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, srcHost=192.168.100.21, dstVdsId=bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, dstHost=192.168.100.22:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 46abcb25 >2014-01-09 13:14:29,567 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [6a43df86] VdsBroker::migrate::Entered (vm_guid=be5348a1-85c7-48dd-a8ad-8a0464643bd8, srcHost=192.168.100.21, dstHost=192.168.100.22:54321, method=online >2014-01-09 13:14:29,568 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [6a43df86] START, MigrateBrokerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, srcHost=192.168.100.21, dstVdsId=bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, dstHost=192.168.100.22:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 4947ddc3 >2014-01-09 13:14:29,576 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [6a43df86] FINISH, MigrateBrokerVDSCommand, log id: 4947ddc3 >2014-01-09 13:14:29,580 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-48) [6a43df86] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 46abcb25 >2014-01-09 13:14:29,584 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) [6a43df86] Correlation ID: 6a43df86, Job ID: 408c670f-6ed4-44e1-a04b-4f697cbe9fe6, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: Apache01, Source: ovirt01, Destination: ovirt02, User: ryan-a). >2014-01-09 13:14:29,593 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-48) [e080ceb] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: fc077d22-f9a1-45e4-839d-840c128a10c6 Type: VM >2014-01-09 13:14:29,607 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-48) [e080ceb] START, MigrateVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=fc077d22-f9a1-45e4-839d-840c128a10c6, srcHost=192.168.100.21, dstVdsId=bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, dstHost=192.168.100.22:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 629a96c >2014-01-09 13:14:29,608 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [e080ceb] VdsBroker::migrate::Entered (vm_guid=fc077d22-f9a1-45e4-839d-840c128a10c6, srcHost=192.168.100.21, dstHost=192.168.100.22:54321, method=online >2014-01-09 13:14:29,609 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [e080ceb] START, MigrateBrokerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=fc077d22-f9a1-45e4-839d-840c128a10c6, srcHost=192.168.100.21, dstVdsId=bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, dstHost=192.168.100.22:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 3af673fc >2014-01-09 13:14:29,619 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [e080ceb] FINISH, MigrateBrokerVDSCommand, log id: 3af673fc >2014-01-09 13:14:29,622 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-48) [e080ceb] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 629a96c >2014-01-09 13:14:29,626 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) [e080ceb] Correlation ID: e080ceb, Job ID: 30e80a35-ba74-474b-81f4-c97d011fec18, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: Finance, Source: ovirt01, Destination: ovirt02, User: ryan-a). >2014-01-09 13:14:29,636 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-48) [32881f7f] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 Type: VM >2014-01-09 13:14:29,654 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-48) [32881f7f] START, MigrateVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=14f2245f-b5a9-48b0-aa78-c8cab04d9be7, srcHost=192.168.100.21, dstVdsId=bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, dstHost=192.168.100.22:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 1663d18d >2014-01-09 13:14:29,656 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [32881f7f] VdsBroker::migrate::Entered (vm_guid=14f2245f-b5a9-48b0-aa78-c8cab04d9be7, srcHost=192.168.100.21, dstHost=192.168.100.22:54321, method=online >2014-01-09 13:14:29,657 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [32881f7f] START, MigrateBrokerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=14f2245f-b5a9-48b0-aa78-c8cab04d9be7, srcHost=192.168.100.21, dstVdsId=bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, dstHost=192.168.100.22:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 7f28e35c >2014-01-09 13:14:29,669 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [32881f7f] FINISH, MigrateBrokerVDSCommand, log id: 7f28e35c >2014-01-09 13:14:29,673 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-48) [32881f7f] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 1663d18d >2014-01-09 13:14:29,679 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) [32881f7f] Correlation ID: 32881f7f, Job ID: 5070e5c7-0f9a-41ce-8d9d-6837491d13c0, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: syslog00, Source: ovirt01, Destination: ovirt02, User: ryan-a). >2014-01-09 13:14:29,688 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-48) [6cbe4c4] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: 33f88d50-30eb-45c7-9850-733da17ed7a4 Type: VM >2014-01-09 13:14:29,702 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-48) [6cbe4c4] START, MigrateVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, srcHost=192.168.100.21, dstVdsId=bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, dstHost=192.168.100.22:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 4fb40d78 >2014-01-09 13:14:29,703 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [6cbe4c4] VdsBroker::migrate::Entered (vm_guid=33f88d50-30eb-45c7-9850-733da17ed7a4, srcHost=192.168.100.21, dstHost=192.168.100.22:54321, method=online >2014-01-09 13:14:29,704 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [6cbe4c4] START, MigrateBrokerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, srcHost=192.168.100.21, dstVdsId=bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, dstHost=192.168.100.22:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 5e9227f8 >2014-01-09 13:14:29,718 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [6cbe4c4] FINISH, MigrateBrokerVDSCommand, log id: 5e9227f8 >2014-01-09 13:14:29,721 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-48) [6cbe4c4] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 4fb40d78 >2014-01-09 13:14:29,725 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) [6cbe4c4] Correlation ID: 6cbe4c4, Job ID: dc43851d-983a-4047-9de7-331abe1cf85a, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: DC-02, Source: ovirt01, Destination: ovirt02, User: ryan-a). >2014-01-09 13:14:32,371 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-24) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 13:14:32,372 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-24) RefreshVmList vm id be5348a1-85c7-48dd-a8ad-8a0464643bd8 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 13:14:32,374 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-24) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 13:14:35,404 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 13:14:35,406 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) RefreshVmList vm id be5348a1-85c7-48dd-a8ad-8a0464643bd8 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 13:14:35,407 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 13:14:38,448 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-33) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 13:14:38,449 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-33) RefreshVmList vm id be5348a1-85c7-48dd-a8ad-8a0464643bd8 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 13:14:38,451 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-33) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 13:14:41,481 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 13:14:41,483 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) RefreshVmList vm id be5348a1-85c7-48dd-a8ad-8a0464643bd8 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 13:14:41,484 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 13:14:49,203 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand] (DefaultQuartzScheduler_Worker-37) Command ListVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:14:49,206 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-37) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 13:14:50,439 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-39) Command SpmStatusVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:14:50,442 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-39) Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 13:14:50,453 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-39) Correlation ID: 1aba1b94, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center CSEnt. Setting Data Center status to Non Responsive (On host ovirt01, Error: Network error during communication with the Host.). >2014-01-09 13:14:52,459 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-39) IrsBroker::Failed::GetStoragePoolInfoVDS due to: SocketTimeoutException: connect timed out >2014-01-09 13:14:54,216 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-27) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:14:54,218 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-27) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 13:14:54,471 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-39) Command SpmStatusVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:14:54,471 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-39) Irs placed on server f2d7a8a9-e77e-4e9a-8031-23a0731407c8 failed. Proceed Failover >2014-01-09 13:14:59,228 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-41) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:14:59,230 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-41) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:04,239 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:04,241 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-50) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:09,249 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:09,251 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-17) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:14,260 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:14,262 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-52) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:14,504 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-39) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:15:14,507 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-39) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:15:14,509 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-39) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:15:14,511 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-39) START, SpmStopVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 5286a2b0 >2014-01-09 13:15:14,512 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-39) SpmStopVDSCommand:: vds ovirt01 is in Connecting status - not performing spm stop, pool id 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 13:15:14,513 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-39) FINISH, SpmStopVDSCommand, log id: 5286a2b0 >2014-01-09 13:15:14,513 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-39) spm stop on spm failed, stopping spm selection! >2014-01-09 13:15:19,272 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:19,273 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-63) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:24,283 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:24,285 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-95) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:29,294 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:29,296 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-69) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:34,306 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:34,308 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-99) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:39,317 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:39,319 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-1) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:44,329 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:44,332 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-76) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:44,544 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-60) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:15:44,547 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-60) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:15:44,549 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-60) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:15:44,550 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-60) START, SpmStopVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 6deef1cb >2014-01-09 13:15:44,551 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-60) SpmStopVDSCommand:: vds ovirt01 is in Connecting status - not performing spm stop, pool id 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 13:15:44,552 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-60) FINISH, SpmStopVDSCommand, log id: 6deef1cb >2014-01-09 13:15:44,553 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-60) spm stop on spm failed, stopping spm selection! >2014-01-09 13:15:49,340 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:49,342 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-59) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:54,351 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:54,353 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-2) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:59,361 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:15:59,363 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-96) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 13:16:04,372 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:16:04,374 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-85) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 13:16:04,614 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-51) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 13:16:04,615 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-51) RefreshVmList vm id be5348a1-85c7-48dd-a8ad-8a0464643bd8 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 13:16:04,616 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-51) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 13:16:09,385 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:16:09,387 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-13) Server failed to respond, vds_id = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds_name = ovirt01, vm_count = 7, spm_status = SPM, non-responsive_timeout (seconds) = 83, error = java.net.SocketTimeoutException: connect timed out >2014-01-09 13:16:09,471 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-13) Correlation ID: null, Call Stack: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:102) > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:28) > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:536) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:492) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.Refresh(VdsUpdateRunTimeInfo.java:343) > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:237) > at sun.reflect.GeneratedMethodAccessor49.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > at java.net.Socket.connect(Socket.java:579) > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) > at com.sun.proxy.$Proxy52.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor167.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:225) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:744) >, Custom Event ID: -1, Message: Host ovirt01 is non responsive. >2014-01-09 13:16:09,483 INFO [org.ovirt.engine.core.bll.VdsEventListener] (pool-6-thread-48) ResourceManager::vdsNotResponding entered for Host f2d7a8a9-e77e-4e9a-8031-23a0731407c8, 192.168.100.21 >2014-01-09 13:16:09,495 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (pool-6-thread-48) Running command: SshSoftFencingCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 13:16:13,444 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-01-09 13:16:13,443 ERROR [org.ovirt.engine.core.bll.SshSoftFencingCommand] (pool-6-thread-48) SSH connection to host 192.168.100.21 failed: org.apache.sshd.common.RuntimeSshException: Failed to get the session.: org.apache.sshd.common.RuntimeSshException: Failed to get the session. > at org.apache.sshd.client.future.DefaultConnectFuture.getSession(DefaultConnectFuture.java:43) > at org.ovirt.engine.core.utils.ssh.SSHClient.connect(SSHClient.java:288) [utils.jar:] > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:59) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeSshSoftFencingCommand(SshSoftFencingCommand.java:73) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeCommand(SshSoftFencingCommand.java:43) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1135) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1220) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1896) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1240) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:366) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:416) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:395) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:636) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view9.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.VdsEventListener$1.run(VdsEventListener.java:132) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:71) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_45] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_45] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_45] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_45] > at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_45] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735) [rt.jar:1.7.0_45] > at org.apache.mina.transport.socket.nio.NioSocketConnector.finishConnect(NioSocketConnector.java:223) [mina-core.jar:] > at org.apache.mina.transport.socket.nio.NioSocketConnector.finishConnect(NioSocketConnector.java:45) [mina-core.jar:] > at org.apache.mina.core.polling.AbstractPollingIoConnector.processConnections(AbstractPollingIoConnector.java:427) [mina-core.jar:] > at org.apache.mina.core.polling.AbstractPollingIoConnector.access$500(AbstractPollingIoConnector.java:62) [mina-core.jar:] > at org.apache.mina.core.polling.AbstractPollingIoConnector$Connector.run(AbstractPollingIoConnector.java:476) [mina-core.jar:] > at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64) [mina-core.jar:] > ... 3 more > >2014-01-09 13:16:13,508 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-48) START, SetVmStatusVDSCommand( vmId = 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, status = Unknown), log id: 74bc3e0b >2014-01-09 13:16:13,510 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-48) FINISH, SetVmStatusVDSCommand, log id: 74bc3e0b >2014-01-09 13:16:13,544 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM VETS was set to the Unknown status. >2014-01-09 13:16:13,546 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (pool-6-thread-48) START, DestroyVmVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, force=true, secondsToWait=0, gracefully=false), log id: 78cabceb >2014-01-09 13:16:14,589 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:16:14,591 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:16:14,593 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:16:14,594 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) spm vds is non responsive, stopping spm selection. >2014-01-09 13:16:18,454 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:16:23,464 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:16:28,475 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:16:33,485 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:16:38,495 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:16:43,506 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-21) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:16:44,629 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:16:44,631 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:16:44,634 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:16:44,634 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) spm vds is non responsive, stopping spm selection. >2014-01-09 13:16:48,518 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-43) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:16:53,528 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:16:58,538 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:17:03,549 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:17:08,560 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-77) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:17:13,571 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:17:14,669 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:17:14,671 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:17:14,674 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:17:14,674 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) spm vds is non responsive, stopping spm selection. >2014-01-09 13:17:18,581 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:17:23,591 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-81) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:17:24,744 WARN [org.ovirt.engine.core.bll.SearchQuery] (ajp--127.0.0.1-8702-6) ResourceManager::searchBusinessObjects - erroneous search text - Vms: Hosts.name = ovirt02 >2014-01-09 13:17:27,735 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-98) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 13:17:27,736 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-98) RefreshVmList vm id be5348a1-85c7-48dd-a8ad-8a0464643bd8 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 13:17:27,737 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-98) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 13:17:27,741 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-48) Lock freed to object EngineLock [exclusiveLocks= key: be5348a1-85c7-48dd-a8ad-8a0464643bd8 value: VM >, sharedLocks= ] >2014-01-09 13:17:27,745 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (pool-6-thread-48) START, DestroyVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, force=true, secondsToWait=0, gracefully=false), log id: 494310bb >2014-01-09 13:17:28,601 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:17:33,611 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-88) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:17:38,621 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:17:43,634 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:17:44,709 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:17:44,711 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:17:44,714 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:17:44,715 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) spm vds is non responsive, stopping spm selection. >2014-01-09 13:17:48,646 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:17:53,223 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (pool-6-thread-48) FINISH, DestroyVDSCommand, log id: 494310bb >2014-01-09 13:17:53,236 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (pool-6-thread-48) FINISH, DestroyVmVDSCommand, return: Down, log id: 78cabceb >2014-01-09 13:17:53,237 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (pool-6-thread-48) Stopped migrating vm: Apache01 on vds: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 >2014-01-09 13:17:53,237 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-48) START, SetVmStatusVDSCommand( vmId = be5348a1-85c7-48dd-a8ad-8a0464643bd8, status = Unknown), log id: 57722a61 >2014-01-09 13:17:53,240 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-48) FINISH, SetVmStatusVDSCommand, log id: 57722a61 >2014-01-09 13:17:53,246 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM Apache01 was set to the Unknown status. >2014-01-09 13:17:53,248 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (pool-6-thread-48) START, DestroyVmVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, force=true, secondsToWait=0, gracefully=false), log id: 13b266ad >2014-01-09 13:17:53,658 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:17:58,670 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:18:03,682 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:18:08,694 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:18:13,709 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:18:14,749 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-85) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:18:14,751 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-85) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:18:14,753 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-85) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:18:14,754 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-85) spm vds is non responsive, stopping spm selection. >2014-01-09 13:18:18,722 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-29) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:18:23,734 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:18:28,746 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:18:33,758 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:18:38,770 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:18:43,782 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-41) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:18:44,782 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-44) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:18:44,784 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-44) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:18:44,787 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-44) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:18:44,787 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-44) spm vds is non responsive, stopping spm selection. >2014-01-09 13:18:48,794 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:18:53,321 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 13:18:53,322 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 13:18:53,327 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-48) Lock freed to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 13:18:53,331 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (pool-6-thread-48) START, DestroyVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, force=true, secondsToWait=0, gracefully=false), log id: 63ff5c6f >2014-01-09 13:18:53,805 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:18:58,817 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:19:03,828 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-77) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:19:08,840 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:19:13,361 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (pool-6-thread-48) Failed in DestroyVDS method >2014-01-09 13:19:13,361 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (pool-6-thread-48) Error code noVM and error message VDSGenericException: VDSErrorException: Failed to DestroyVDS, error = Virtual machine does not exist >2014-01-09 13:19:13,363 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (pool-6-thread-48) Command org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand return value > StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=1, mMessage=Virtual machine does not exist]] >2014-01-09 13:19:13,363 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (pool-6-thread-48) HostName = ovirt02 >2014-01-09 13:19:13,364 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (pool-6-thread-48) Command DestroyVDS execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to DestroyVDS, error = Virtual machine does not exist >2014-01-09 13:19:13,365 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (pool-6-thread-48) FINISH, DestroyVDSCommand, log id: 63ff5c6f >2014-01-09 13:19:13,365 ERROR [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (pool-6-thread-48) VDS::destroy Failed destroying vm 33f88d50-30eb-45c7-9850-733da17ed7a4 in vds = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 : ovirt02, error = org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to DestroyVDS, error = Virtual machine does not exist >2014-01-09 13:19:13,367 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (pool-6-thread-48) FINISH, DestroyVmVDSCommand, log id: 13b266ad >2014-01-09 13:19:13,367 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (pool-6-thread-48) Could not stop migrating vm: DC-02 on vds: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, Error: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to DestroyVDS, error = Virtual machine does not exist (Failed with error noVM and code 1) >2014-01-09 13:19:13,369 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-48) START, SetVmStatusVDSCommand( vmId = 33f88d50-30eb-45c7-9850-733da17ed7a4, status = Unknown), log id: 8b075f7 >2014-01-09 13:19:13,372 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-48) FINISH, SetVmStatusVDSCommand, log id: 8b075f7 >2014-01-09 13:19:13,379 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM DC-02 was set to the Unknown status. >2014-01-09 13:19:13,380 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-48) START, SetVmStatusVDSCommand( vmId = b6972001-2bd6-4cea-9c15-c046674754d4, status = Unknown), log id: 123eb574 >2014-01-09 13:19:13,382 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-48) FINISH, SetVmStatusVDSCommand, log id: 123eb574 >2014-01-09 13:19:13,388 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM DC-01 was set to the Unknown status. >2014-01-09 13:19:13,390 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (pool-6-thread-48) START, DestroyVmVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=14f2245f-b5a9-48b0-aa78-c8cab04d9be7, force=true, secondsToWait=0, gracefully=false), log id: 67995388 >2014-01-09 13:19:13,853 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:19:14,820 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:19:14,822 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:19:14,825 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:19:14,825 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) spm vds is non responsive, stopping spm selection. >2014-01-09 13:19:18,865 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:19:23,877 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-89) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:19:28,890 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:19:33,902 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:19:38,913 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:19:43,925 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-57) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:19:44,859 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:19:44,862 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:19:44,864 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:19:44,864 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) spm vds is non responsive, stopping spm selection. >2014-01-09 13:19:48,938 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:19:53,950 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:19:58,961 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:20:03,973 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:20:08,986 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:20:13,448 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-3) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 status = Paused on vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 13:20:13,449 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-3) RefreshVmList vm id 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 status = NotResponding on vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 13:20:13,453 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-48) Lock freed to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 13:20:13,457 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (pool-6-thread-48) START, DestroyVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=14f2245f-b5a9-48b0-aa78-c8cab04d9be7, force=true, secondsToWait=0, gracefully=false), log id: 17a855e >2014-01-09 13:20:13,999 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:20:14,883 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:20:14,885 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:20:14,887 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:20:14,888 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) spm vds is non responsive, stopping spm selection. >2014-01-09 13:20:19,011 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:20:24,023 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:20:29,035 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:20:34,045 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-43) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:20:39,057 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:20:44,068 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:20:44,922 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-56) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:20:44,924 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-56) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:20:44,926 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-56) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:20:44,927 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-56) spm vds is non responsive, stopping spm selection. >2014-01-09 13:20:49,078 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:20:54,090 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:20:59,102 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:21:04,114 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:21:09,127 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:21:14,139 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:21:14,943 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-75) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:21:14,945 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-75) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:21:14,948 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-75) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:21:14,948 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-75) spm vds is non responsive, stopping spm selection. >2014-01-09 13:21:19,150 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-89) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:21:24,162 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:21:29,174 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:21:30,114 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (pool-6-thread-48) FINISH, DestroyVDSCommand, log id: 17a855e >2014-01-09 13:21:30,125 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (pool-6-thread-48) FINISH, DestroyVmVDSCommand, return: Down, log id: 67995388 >2014-01-09 13:21:30,126 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (pool-6-thread-48) Stopped migrating vm: syslog00 on vds: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 >2014-01-09 13:21:30,127 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-48) START, SetVmStatusVDSCommand( vmId = 14f2245f-b5a9-48b0-aa78-c8cab04d9be7, status = Unknown), log id: 47441c6c >2014-01-09 13:21:30,129 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-48) FINISH, SetVmStatusVDSCommand, log id: 47441c6c >2014-01-09 13:21:30,136 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM syslog00 was set to the Unknown status. >2014-01-09 13:21:30,138 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (pool-6-thread-48) START, DestroyVmVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=fc077d22-f9a1-45e4-839d-840c128a10c6, force=true, secondsToWait=0, gracefully=false), log id: 25741e77 >2014-01-09 13:21:34,186 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:21:39,199 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:21:44,210 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:21:44,983 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-7) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:21:44,985 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-7) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:21:44,987 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-7) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:21:44,987 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-7) spm vds is non responsive, stopping spm selection. >2014-01-09 13:21:49,222 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:21:54,233 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:21:59,245 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:22:04,257 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:22:09,268 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:22:10,204 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-9) RefreshVmList vm id fc077d22-f9a1-45e4-839d-840c128a10c6 status = Paused on vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 13:22:10,212 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-48) Lock freed to object EngineLock [exclusiveLocks= key: fc077d22-f9a1-45e4-839d-840c128a10c6 value: VM >, sharedLocks= ] >2014-01-09 13:22:10,216 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (pool-6-thread-48) START, DestroyVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=fc077d22-f9a1-45e4-839d-840c128a10c6, force=true, secondsToWait=0, gracefully=false), log id: 772c6771 >2014-01-09 13:22:14,282 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:22:15,022 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-26) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:22:15,024 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-26) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:22:15,026 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-26) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:22:15,027 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-26) spm vds is non responsive, stopping spm selection. >2014-01-09 13:22:19,293 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-27) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:22:24,306 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-21) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:22:29,318 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:22:34,330 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:22:39,340 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:22:44,350 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:22:45,056 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:22:45,058 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:22:45,061 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:22:45,061 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) spm vds is non responsive, stopping spm selection. >2014-01-09 13:22:49,359 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-77) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:22:54,372 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:22:59,383 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:23:04,395 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-81) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:23:09,406 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:23:14,417 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:23:15,095 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:23:15,097 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:23:15,099 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:23:15,099 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) spm vds is non responsive, stopping spm selection. >2014-01-09 13:23:19,429 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:23:24,441 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:23:26,363 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (pool-6-thread-48) FINISH, DestroyVDSCommand, log id: 772c6771 >2014-01-09 13:23:26,371 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (pool-6-thread-48) FINISH, DestroyVmVDSCommand, return: Down, log id: 25741e77 >2014-01-09 13:23:26,371 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (pool-6-thread-48) Stopped migrating vm: Finance on vds: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 >2014-01-09 13:23:26,372 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-48) START, SetVmStatusVDSCommand( vmId = fc077d22-f9a1-45e4-839d-840c128a10c6, status = Unknown), log id: 5fac486d >2014-01-09 13:23:26,375 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-48) FINISH, SetVmStatusVDSCommand, log id: 5fac486d >2014-01-09 13:23:26,385 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM Finance was set to the Unknown status. >2014-01-09 13:23:26,386 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-48) START, SetVmStatusVDSCommand( vmId = 872c3dea-6984-48fa-99c3-354fd367c3f9, status = Unknown), log id: 2eb06e9f >2014-01-09 13:23:26,388 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-48) FINISH, SetVmStatusVDSCommand, log id: 2eb06e9f >2014-01-09 13:23:26,395 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM External_Email was set to the Unknown status. >2014-01-09 13:23:26,396 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (pool-6-thread-48) Fence failed on vds ovirt01 which is spm of pool 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d - moving pool to non operational >2014-01-09 13:23:26,398 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (pool-6-thread-48) Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 13:23:26,406 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) Correlation ID: 28db78bd, Call Stack: null, Custom Event ID: -1, Message: Fencing failed on Storage Pool Manager ovirt01 for Data Center CSEnt. Setting status to Non-Operational. >2014-01-09 13:23:26,407 ERROR [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (pool-6-thread-48) Failed to run Fence script on vds:ovirt01, VMs moved to UnKnown instead. >2014-01-09 13:23:26,412 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host ovirt01 became non responsive. It has no power management configured. Please check the host status, manually reboot it, and click "Confirm Host Has Been Rebooted" >2014-01-09 13:23:26,413 WARN [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (pool-6-thread-48) CanDoAction of action VdsNotRespondingTreatment failed. Reasons:VDS_FENCE_DISABLED >2014-01-09 13:23:29,452 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:23:34,462 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:23:39,473 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:23:44,485 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:23:45,133 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-70) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:23:45,134 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-70) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:23:45,137 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-70) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:23:45,137 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-70) spm vds is non responsive, stopping spm selection. >2014-01-09 13:23:49,497 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-55) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:23:54,507 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:23:59,519 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:24:04,531 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:24:09,542 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:24:14,555 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:24:19,565 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-43) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:24:24,576 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:24:29,588 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-56) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:24:34,600 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:24:39,611 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:24:44,623 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:24:49,635 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:24:54,647 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:24:59,658 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:25:04,670 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:25:09,682 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:25:12,640 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-47) Attempting to update VMs/Templates Ovf. >2014-01-09 13:25:14,693 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:25:19,705 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:25:23,175 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-38) Start refreshing all users data >2014-01-09 13:25:24,716 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:25:29,727 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:25:34,739 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:25:39,750 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:25:44,762 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:25:49,774 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:25:54,786 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:25:59,798 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:26:03,220 ERROR [org.ovirt.engine.core.bll.adbroker.LdapSearchExceptionHandler] (DefaultQuartzScheduler_Worker-38) Error in communicating with LDAP server cydc-02.cytechservices.local:389; nested exception is javax.naming.CommunicationException: cydc-02.cytechservices.local:389 [Root exception is java.net.UnknownHostException: cydc-02.cytechservices.local] >2014-01-09 13:26:03,222 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-38) Failed ldap search server LDAP://cydc-02.cytechservices.local:389 using user ryan-a@CYTECHSERVICES.LOCAL due to cydc-02.cytechservices.local:389; nested exception is javax.naming.CommunicationException: cydc-02.cytechservices.local:389 [Root exception is java.net.UnknownHostException: cydc-02.cytechservices.local]. We should try the next server >2014-01-09 13:26:04,810 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:26:09,823 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:26:14,836 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-77) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:26:19,848 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:26:24,860 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:26:26,404 INFO [org.ovirt.engine.core.bll.storage.StoragePoolStatusHandler] (DefaultQuartzScheduler_Worker-81) Moving data center CSEnt with Id 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d to status Problematic from status NotOperational on a one time basis to try to recover >2014-01-09 13:26:26,407 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-81) Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 13:26:26,414 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-81) Correlation ID: 7f2ce4a5, Call Stack: null, Custom Event ID: -1, Message: Try to recover Data Center CSEnt. Setting status to Non Responsive. >2014-01-09 13:26:29,871 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:26:34,883 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:26:39,893 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:26:43,266 ERROR [org.ovirt.engine.core.bll.adbroker.LdapSearchExceptionHandler] (DefaultQuartzScheduler_Worker-38) Error in communicating with LDAP server cydc-01.cytechservices.local:389; nested exception is javax.naming.CommunicationException: cydc-01.cytechservices.local:389 [Root exception is java.net.UnknownHostException: cydc-01.cytechservices.local] >2014-01-09 13:26:43,268 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-38) Failed ldap search server LDAP://cydc-01.cytechservices.local:389 using user ryan-a@CYTECHSERVICES.LOCAL due to cydc-01.cytechservices.local:389; nested exception is javax.naming.CommunicationException: cydc-01.cytechservices.local:389 [Root exception is java.net.UnknownHostException: cydc-01.cytechservices.local]. We should try the next server >2014-01-09 13:26:43,270 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-38) User Ryan not found in directory server, its status switched to InActive >2014-01-09 13:26:43,272 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-38) User Ryan not found in directory server, its status switched to InActive >2014-01-09 13:26:43,274 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-38) User Patrick not found in directory server, its status switched to InActive >2014-01-09 13:26:43,275 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-38) User Patrick not found in directory server, its status switched to InActive >2014-01-09 13:26:43,276 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-38) User Chris not found in directory server, its status switched to InActive >2014-01-09 13:26:44,905 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:26:49,916 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:26:54,928 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:26:55,191 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:26:55,193 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:26:55,195 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:26:55,195 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) spm vds is non responsive, stopping spm selection. >2014-01-09 13:26:59,939 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:27:04,949 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:27:09,960 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:27:14,972 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:27:19,984 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:27:24,994 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:27:25,229 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-32) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:27:25,231 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-32) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:27:25,233 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-32) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:27:25,234 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-32) spm vds is non responsive, stopping spm selection. >2014-01-09 13:27:30,005 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:27:35,016 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:27:40,027 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:27:45,039 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:27:50,050 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:27:55,061 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:27:55,264 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:27:55,266 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:27:55,269 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:27:55,269 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) spm vds is non responsive, stopping spm selection. >2014-01-09 13:28:00,073 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-41) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:28:05,084 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:28:10,095 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:28:15,109 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:28:20,121 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:28:25,132 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:28:25,298 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:28:25,300 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:28:25,302 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:28:25,302 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) spm vds is non responsive, stopping spm selection. >2014-01-09 13:28:30,144 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:28:35,155 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:28:40,167 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:28:45,177 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:28:50,189 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:28:55,201 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:28:55,318 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-59) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:28:55,319 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-59) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:28:55,321 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-59) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:28:55,322 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-59) spm vds is non responsive, stopping spm selection. >2014-01-09 13:29:00,211 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:29:05,223 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:29:10,235 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:29:15,248 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:29:20,259 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:29:25,271 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:29:25,355 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-11) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:29:25,357 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-11) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:29:25,359 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-11) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:29:25,359 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-11) spm vds is non responsive, stopping spm selection. >2014-01-09 13:29:30,282 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:29:35,293 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:29:40,304 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:29:45,315 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:29:50,327 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:29:55,339 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:29:55,382 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:29:55,383 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:29:55,386 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:29:55,386 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) spm vds is non responsive, stopping spm selection. >2014-01-09 13:30:00,351 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:30:05,362 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:30:10,374 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:30:15,388 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:30:20,400 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:30:25,412 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:30:25,426 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-69) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:30:25,428 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-69) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:30:25,430 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-69) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:30:25,430 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-69) spm vds is non responsive, stopping spm selection. >2014-01-09 13:30:30,424 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:30:35,435 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:30:40,446 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:30:45,458 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:30:50,468 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:30:55,452 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-96) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:30:55,454 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-96) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:30:55,457 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-96) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:30:55,457 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-96) spm vds is non responsive, stopping spm selection. >2014-01-09 13:30:55,478 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:31:00,490 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:31:05,502 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:31:10,515 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:31:15,526 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:31:20,538 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-43) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:31:25,483 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-32) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:31:25,485 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-32) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:31:25,487 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-32) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:31:25,488 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-32) spm vds is non responsive, stopping spm selection. >2014-01-09 13:31:25,550 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:31:30,561 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-29) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:31:35,574 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:31:40,585 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-77) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:31:45,597 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:31:50,608 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:31:55,510 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:31:55,512 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:31:55,514 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:31:55,515 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) spm vds is non responsive, stopping spm selection. >2014-01-09 13:31:55,620 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-81) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:32:00,632 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:32:05,644 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:32:10,655 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:32:15,668 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:32:20,679 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:32:25,547 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-60) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:32:25,549 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-60) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:32:25,552 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-60) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:32:25,552 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-60) spm vds is non responsive, stopping spm selection. >2014-01-09 13:32:25,690 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:32:30,702 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:32:35,713 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:32:40,725 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:32:45,737 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:32:50,749 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:32:55,579 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-34) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:32:55,581 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-34) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:32:55,583 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-34) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:32:55,584 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-34) spm vds is non responsive, stopping spm selection. >2014-01-09 13:32:55,760 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:33:00,772 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:33:05,784 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:33:10,795 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-21) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:33:15,806 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:33:20,818 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:33:25,624 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-92) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:33:25,626 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-92) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:33:25,628 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-92) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:33:25,628 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-92) spm vds is non responsive, stopping spm selection. >2014-01-09 13:33:25,828 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-27) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:33:30,839 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-77) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:33:35,851 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:33:40,862 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:33:45,874 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-41) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:33:50,886 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:33:55,652 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-14) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:33:55,654 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-14) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:33:55,656 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-14) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:33:55,657 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-14) spm vds is non responsive, stopping spm selection. >2014-01-09 13:33:55,898 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:34:00,910 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:34:05,922 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:34:10,934 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:34:15,948 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:34:20,959 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:34:25,690 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:34:25,692 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:34:25,694 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:34:25,695 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) spm vds is non responsive, stopping spm selection. >2014-01-09 13:34:25,970 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:34:30,982 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:34:35,993 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:34:41,004 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:34:46,016 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:34:46,456 ERROR [org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy] (ajp--127.0.0.1-8702-5) Kerberos error: Cannot locate KDC >2014-01-09 13:34:46,457 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (ajp--127.0.0.1-8702-5) Failed ldap search server LDAP://cydc-02.cytechservices.local:389 using user ryan-a@CYTECHSERVICES.LOCAL due to Kerberos error. Please check log for further details.. We should not try the next server >2014-01-09 13:34:51,033 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:34:55,728 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-94) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:34:55,730 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-94) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:34:55,732 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-94) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:34:55,733 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-94) spm vds is non responsive, stopping spm selection. >2014-01-09 13:34:56,045 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:35:01,055 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:35:06,066 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-43) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:35:11,080 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:35:16,091 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-27) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:35:21,107 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:35:25,765 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-17) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:35:25,767 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-17) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:35:25,769 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-17) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:35:25,770 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-17) spm vds is non responsive, stopping spm selection. >2014-01-09 13:35:26,119 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:35:31,132 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:35:36,143 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:35:41,154 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:35:46,166 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:35:46,508 ERROR [org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy] (ajp--127.0.0.1-8702-5) Kerberos error: Cannot locate KDC >2014-01-09 13:35:46,509 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (ajp--127.0.0.1-8702-5) Failed ldap search server LDAP://cydc-01.cytechservices.local:389 using user ryan-a@CYTECHSERVICES.LOCAL due to Kerberos error. Please check log for further details.. We should not try the next server >2014-01-09 13:35:46,510 ERROR [org.ovirt.engine.core.bll.adbroker.LdapAuthenticateUserCommand] (ajp--127.0.0.1-8702-5) Failed authenticating user: ryan-a to domain cytechservices.local. Ldap Query Type is getUserByName >2014-01-09 13:35:46,510 ERROR [org.ovirt.engine.core.bll.adbroker.LdapAuthenticateUserCommand] (ajp--127.0.0.1-8702-5) Kerberos error. Please check log for further details. >2014-01-09 13:35:46,511 ERROR [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-5) USER_FAILED_TO_AUTHENTICATE : ryan-a >2014-01-09 13:35:46,511 WARN [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-5) CanDoAction of action LoginAdminUser failed. Reasons:USER_FAILED_TO_AUTHENTICATE >2014-01-09 13:35:51,179 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:35:55,803 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-42) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:35:55,805 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-42) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:35:55,807 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-42) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:35:55,808 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-42) spm vds is non responsive, stopping spm selection. >2014-01-09 13:35:56,189 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:36:01,201 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:36:06,212 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:36:11,225 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:36:16,238 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-55) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:36:21,250 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:36:25,843 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-51) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:36:25,844 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-51) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:36:25,847 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-51) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:36:25,847 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-51) spm vds is non responsive, stopping spm selection. >2014-01-09 13:36:26,262 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:36:31,274 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:36:36,285 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:36:41,296 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:36:46,308 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:36:51,320 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:36:55,880 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-36) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:36:55,882 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-36) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:36:55,885 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-36) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:36:55,885 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-36) spm vds is non responsive, stopping spm selection. >2014-01-09 13:36:56,332 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:37:01,343 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:37:06,354 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:37:11,365 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-77) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:37:16,377 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:37:21,388 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:37:25,919 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:37:25,921 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:37:25,923 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:37:25,924 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) spm vds is non responsive, stopping spm selection. >2014-01-09 13:37:26,400 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:37:31,411 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:37:36,422 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:37:41,434 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:37:46,445 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:37:51,457 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:37:55,959 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-91) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:37:55,961 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-91) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:37:55,963 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-91) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:37:55,963 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-91) spm vds is non responsive, stopping spm selection. >2014-01-09 13:37:56,469 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:38:01,479 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:38:06,490 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:38:11,502 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:38:16,515 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:38:21,526 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-89) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:38:25,985 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:38:25,987 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:38:25,989 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:38:25,990 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) spm vds is non responsive, stopping spm selection. >2014-01-09 13:38:26,538 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:38:30,514 INFO [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-10) Running command: LoginAdminUserCommand internal: false. >2014-01-09 13:38:30,518 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2014-01-09 13:38:31,550 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:38:36,562 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:38:41,573 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:38:46,584 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:38:51,596 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:38:56,259 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:38:56,261 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:38:56,263 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:38:56,264 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) spm vds is non responsive, stopping spm selection. >2014-01-09 13:38:56,606 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:39:01,616 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:39:06,627 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:39:11,639 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-56) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:39:16,650 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:39:21,661 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:39:26,298 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-87) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:39:26,300 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-87) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:39:26,302 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-87) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:39:26,303 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-87) spm vds is non responsive, stopping spm selection. >2014-01-09 13:39:26,673 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:39:31,685 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:39:36,697 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:39:41,709 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:39:46,721 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:39:51,732 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:39:56,336 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-2) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:39:56,338 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-2) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:39:56,340 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-2) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:39:56,341 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-2) spm vds is non responsive, stopping spm selection. >2014-01-09 13:39:56,744 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:40:01,756 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:40:06,768 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:40:11,780 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:40:16,793 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:40:21,804 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:40:26,383 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-23) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:40:26,385 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-23) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:40:26,387 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-23) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:40:26,388 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-23) spm vds is non responsive, stopping spm selection. >2014-01-09 13:40:26,815 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:40:30,652 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp--127.0.0.1-8702-4) Running command: LoginUserCommand internal: false. >2014-01-09 13:40:30,656 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2014-01-09 13:40:31,826 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:40:36,837 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:40:41,848 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:40:46,859 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:40:51,871 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:40:56,428 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-58) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:40:56,430 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-58) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:40:56,433 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-58) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:40:56,433 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-58) spm vds is non responsive, stopping spm selection. >2014-01-09 13:40:56,882 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:41:01,894 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:41:06,905 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:41:11,918 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:41:16,929 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:41:21,940 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:41:26,467 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:41:26,469 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:41:26,471 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:41:26,472 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) spm vds is non responsive, stopping spm selection. >2014-01-09 13:41:26,951 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:41:31,962 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:41:36,974 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:41:41,985 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:41:46,997 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:41:52,009 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:41:56,506 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-59) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:41:56,508 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-59) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:41:56,510 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-59) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:41:56,510 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-59) spm vds is non responsive, stopping spm selection. >2014-01-09 13:41:57,021 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:42:02,033 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:42:07,044 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:42:12,056 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:42:17,068 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:42:22,079 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:42:26,544 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-26) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:42:26,546 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-26) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:42:26,548 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-26) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:42:26,548 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-26) spm vds is non responsive, stopping spm selection. >2014-01-09 13:42:27,091 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-29) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:42:32,102 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:42:37,113 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-21) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:42:42,125 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:42:47,137 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:42:52,149 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-56) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:42:56,573 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-44) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:42:56,575 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-44) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:42:56,577 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-44) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:42:56,578 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-44) spm vds is non responsive, stopping spm selection. >2014-01-09 13:42:57,161 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:43:02,171 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:43:07,181 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-87) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:43:12,192 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:43:17,203 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:43:22,214 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:43:26,612 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-71) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:43:26,614 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-71) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:43:26,616 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-71) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:43:26,616 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-71) spm vds is non responsive, stopping spm selection. >2014-01-09 13:43:27,226 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:43:32,237 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:43:37,248 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:43:42,260 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:43:47,271 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:43:52,283 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:43:56,662 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:43:56,664 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:43:56,667 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:43:56,667 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) spm vds is non responsive, stopping spm selection. >2014-01-09 13:43:57,295 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:44:02,305 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:44:07,316 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:44:12,328 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:44:17,341 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:44:22,353 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:44:26,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:44:26,701 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:44:26,703 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:44:26,704 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) spm vds is non responsive, stopping spm selection. >2014-01-09 13:44:27,364 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:44:32,376 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:44:37,388 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:44:42,400 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:44:47,411 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:44:52,422 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:44:56,733 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-58) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:44:56,735 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-58) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:44:56,737 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-58) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:44:56,737 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-58) spm vds is non responsive, stopping spm selection. >2014-01-09 13:44:57,440 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-87) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:45:02,452 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:45:07,469 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:45:12,480 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:45:17,491 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:45:22,503 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:45:26,792 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:45:26,794 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:45:26,796 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:45:26,797 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) spm vds is non responsive, stopping spm selection. >2014-01-09 13:45:27,515 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:45:32,526 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:45:37,537 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:45:42,549 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:45:47,562 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:45:52,574 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:45:56,834 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:45:56,836 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:45:56,838 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:45:56,839 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) spm vds is non responsive, stopping spm selection. >2014-01-09 13:45:57,586 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:46:02,597 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:46:07,608 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:46:12,621 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:46:17,634 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-21) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:46:22,645 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:46:26,890 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-29) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:46:26,892 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-29) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:46:26,895 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-29) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:46:26,896 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-29) spm vds is non responsive, stopping spm selection. >2014-01-09 13:46:27,657 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-81) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:46:32,669 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:46:37,682 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:46:42,692 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:46:47,704 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:46:52,715 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:46:56,928 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-25) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:46:56,930 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-25) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:46:56,933 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-25) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:46:56,933 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-25) spm vds is non responsive, stopping spm selection. >2014-01-09 13:46:57,725 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:47:02,737 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:47:07,749 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:47:12,759 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:47:17,769 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:47:22,781 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:47:26,967 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-7) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:47:26,972 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-7) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:47:26,975 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-7) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:47:26,976 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-7) spm vds is non responsive, stopping spm selection. >2014-01-09 13:47:27,791 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:47:32,803 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:47:37,813 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:47:42,824 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:47:47,836 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:47:52,847 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:47:57,009 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-55) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:47:57,011 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-55) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:47:57,014 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-55) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:47:57,014 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-55) spm vds is non responsive, stopping spm selection. >2014-01-09 13:47:57,859 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:48:02,871 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:48:07,881 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:48:12,891 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:48:17,905 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-56) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:48:22,916 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-88) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:48:27,048 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:48:27,050 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:48:27,052 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:48:27,052 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) spm vds is non responsive, stopping spm selection. >2014-01-09 13:48:27,927 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:48:32,939 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:48:37,950 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-87) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:48:42,961 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:48:47,973 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:48:52,983 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:48:57,084 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-20) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:48:57,086 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-20) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:48:57,089 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-20) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:48:57,089 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-20) spm vds is non responsive, stopping spm selection. >2014-01-09 13:48:57,999 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:49:03,011 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:49:08,023 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:49:13,035 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:49:18,047 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-89) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:49:23,058 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:49:27,124 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:49:27,126 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:49:27,129 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:49:27,130 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) spm vds is non responsive, stopping spm selection. >2014-01-09 13:49:28,074 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:49:33,086 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:49:38,098 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:49:43,110 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:49:48,121 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-33) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:49:53,132 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:49:57,170 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:49:57,172 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:49:57,174 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:49:57,175 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) spm vds is non responsive, stopping spm selection. >2014-01-09 13:49:58,145 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-21) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:50:03,155 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:50:08,166 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:50:13,178 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:50:18,191 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:50:23,203 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:50:27,222 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:50:27,224 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:50:27,226 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:50:27,227 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) spm vds is non responsive, stopping spm selection. >2014-01-09 13:50:28,214 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-87) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:50:33,225 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:50:38,237 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:50:43,249 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:50:48,261 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:50:53,272 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:50:57,265 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-99) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:50:57,267 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-99) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:50:57,269 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-99) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:50:57,270 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-99) spm vds is non responsive, stopping spm selection. >2014-01-09 13:50:58,284 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:51:03,295 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:51:08,307 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:51:13,319 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:51:18,331 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:51:23,343 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:51:27,327 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-22) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:51:27,329 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-22) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:51:27,331 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-22) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:51:27,332 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-22) spm vds is non responsive, stopping spm selection. >2014-01-09 13:51:28,354 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:51:33,365 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 13:51:37,263 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:51:40,273 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:51:43,281 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-21) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:51:46,290 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:51:49,301 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:51:52,310 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-29) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:51:55,320 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:51:57,356 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:51:57,358 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:51:57,360 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:51:57,361 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) spm vds is non responsive, stopping spm selection. >2014-01-09 13:51:58,330 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:52:01,339 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:52:04,348 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:52:07,356 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:52:10,366 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:52:13,375 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:52:16,384 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-87) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:52:19,393 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:52:22,403 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:52:25,413 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:52:27,381 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-80) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:52:27,383 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-80) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:52:27,385 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-80) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:52:27,386 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-80) spm vds is non responsive, stopping spm selection. >2014-01-09 13:52:28,423 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:52:31,431 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:52:34,441 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:52:37,450 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:52:40,459 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:52:43,469 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:52:46,479 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:52:49,488 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-57) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:52:52,498 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:52:55,508 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:52:57,420 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-53) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:52:57,422 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-53) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:52:57,425 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-53) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:52:57,425 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-53) spm vds is non responsive, stopping spm selection. >2014-01-09 13:52:58,516 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:53:01,526 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-43) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:53:04,534 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:53:07,544 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:53:10,553 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:53:13,563 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:53:16,572 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:53:19,582 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:53:22,592 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-21) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:53:25,602 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:53:27,448 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-22) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:53:27,450 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-22) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:53:27,452 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-22) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:53:27,453 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-22) spm vds is non responsive, stopping spm selection. >2014-01-09 13:53:28,612 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:53:31,621 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:53:34,631 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:53:37,640 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:53:40,649 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:53:43,659 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:53:46,668 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:53:49,678 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-55) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:53:52,689 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:53:55,697 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:53:57,478 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:53:57,480 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:53:57,482 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:53:57,483 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) spm vds is non responsive, stopping spm selection. >2014-01-09 13:53:58,708 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 13:54:22,712 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-91) START, GetHardwareInfoVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01]), log id: 5e05af83 >2014-01-09 13:54:27,504 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-80) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:54:27,506 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-80) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:54:27,508 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-80) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:54:27,509 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-80) spm vds is non responsive, stopping spm selection. >2014-01-09 13:54:42,885 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-91) FINISH, GetHardwareInfoVDSCommand, log id: 5e05af83 >2014-01-09 13:54:42,900 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-91) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host ovirt01. >2014-01-09 13:54:57,543 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-57) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:54:57,545 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-57) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:54:57,547 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-57) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:54:57,548 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-57) spm vds is non responsive, stopping spm selection. >2014-01-09 13:55:02,928 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-91) vm DC-02 running in db and not running in vds - add to rerun treatment. vds ovirt01 >2014-01-09 13:55:02,929 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-91) vm VETS running in db and not running in vds - add to rerun treatment. vds ovirt01 >2014-01-09 13:55:02,930 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-91) vm DC-01 running in db and not running in vds - add to rerun treatment. vds ovirt01 >2014-01-09 13:55:02,931 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-91) vm External_Email running in db and not running in vds - add to rerun treatment. vds ovirt01 >2014-01-09 13:55:02,932 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-91) Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 13:55:02,934 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-91) Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 13:55:03,740 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-91) START, ConnectStorageServerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = ISCSI, connectionList = [{ id: 623aa506-12d1-4437-8e81-f39b2317f3dd, connection: 192.168.101.101, iqn: iqn.2013-11.local.cytechservices.n16000pro:iscsi.hosts-tgt00.raid0, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 5bac7bd0 >2014-01-09 13:55:26,749 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-91) FINISH, ConnectStorageServerVDSCommand, return: {623aa506-12d1-4437-8e81-f39b2317f3dd=0}, log id: 5bac7bd0 >2014-01-09 13:55:26,751 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-91) Host ovirt01 storage connection was succeeded >2014-01-09 13:55:26,752 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-91) START, ConnectStorageServerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 210997b7-9098-48d6-8e97-9689085aa4b0, connection: 192.168.101.101:/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: AUTO, nfsRetrans: null, nfsTimeo: null };]), log id: 1111b7fc >2014-01-09 13:55:27,582 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-12) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:55:27,584 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-12) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:55:27,586 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-12) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:55:27,587 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-12) spm vds is non responsive, stopping spm selection. >2014-01-09 13:55:47,103 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-91) FINISH, ConnectStorageServerVDSCommand, return: {210997b7-9098-48d6-8e97-9689085aa4b0=0}, log id: 1111b7fc >2014-01-09 13:55:47,104 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-91) Host ovirt01 storage connection was succeeded >2014-01-09 13:55:47,106 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-91) START, ConnectStorageServerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 4e5d4030-1bd2-4487-b08f-2135eba3f11f, connection: 192.168.101.101:/vms, iqn: null, vfsType: null, mountOptions: null, nfsVersion: V4, nfsRetrans: null, nfsTimeo: null };]), log id: 2348cf99 >2014-01-09 13:55:57,622 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-52) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:55:57,624 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-52) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:55:57,626 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-52) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:55:57,627 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-52) spm vds is non responsive, stopping spm selection. >2014-01-09 13:56:07,282 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-91) FINISH, ConnectStorageServerVDSCommand, return: {4e5d4030-1bd2-4487-b08f-2135eba3f11f=0}, log id: 2348cf99 >2014-01-09 13:56:07,284 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-91) Host ovirt01 storage connection was succeeded >2014-01-09 13:56:07,287 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-6-thread-48) START, ConnectStoragePoolVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, vds_spm_id = 1, masterDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, masterVersion = 2), log id: 4893854c >2014-01-09 13:56:27,662 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-98) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:56:27,664 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-98) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:56:27,667 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-98) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:56:27,667 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-98) spm vds is non responsive, stopping spm selection. >2014-01-09 13:56:33,709 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-6-thread-48) FINISH, ConnectStoragePoolVDSCommand, log id: 4893854c >2014-01-09 13:56:53,748 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-91) START, SetMOMPolicyParametersVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8), log id: 3a7ffa65 >2014-01-09 13:56:57,701 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-77) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:56:57,703 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-77) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:56:57,706 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-77) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:56:57,706 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-77) spm vds is non responsive, stopping spm selection. >2014-01-09 13:57:13,860 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-91) FINISH, SetMOMPolicyParametersVDSCommand, log id: 3a7ffa65 >2014-01-09 13:57:13,865 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-91) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host ovirt01. >2014-01-09 13:57:13,873 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-91) Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 13:57:13,876 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-91) Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 13:57:13,884 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-91) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Highly Available VM DC-02 failed. It will be restarted automatically. >2014-01-09 13:57:13,886 INFO [org.ovirt.engine.core.bll.VdsEventListener] (DefaultQuartzScheduler_Worker-91) Highly Available VM went down. Attempting to restart. VM Name: DC-02, VM Id:33f88d50-30eb-45c7-9850-733da17ed7a4 >2014-01-09 13:57:13,903 INFO [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-91) Lock Acquired to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 13:57:13,919 WARN [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-91) CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,VM_CANNOT_RUN_FROM_CD_WITHOUT_ACTIVE_STORAGE_DOMAIN_ISO >2014-01-09 13:57:13,920 INFO [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-91) Lock freed to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 13:57:13,925 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-91) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Restart of the Highly Available VM DC-02 failed. >2014-01-09 13:57:13,931 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-91) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Highly Available VM VETS failed. It will be restarted automatically. >2014-01-09 13:57:13,932 INFO [org.ovirt.engine.core.bll.VdsEventListener] (DefaultQuartzScheduler_Worker-91) Highly Available VM went down. Attempting to restart. VM Name: VETS, VM Id:60ace091-8a4e-4047-9ac0-c0ea95e3c5fa >2014-01-09 13:57:13,947 INFO [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-91) Lock Acquired to object EngineLock [exclusiveLocks= key: 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa value: VM >, sharedLocks= ] >2014-01-09 13:57:13,958 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (DefaultQuartzScheduler_Worker-91) START, IsVmDuringInitiatingVDSCommand( vmId = 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa), log id: 7617eb94 >2014-01-09 13:57:13,959 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (DefaultQuartzScheduler_Worker-91) FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 7617eb94 >2014-01-09 13:57:13,971 INFO [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-91) Running command: RunVmCommand internal: true. Entities affected : ID: 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa Type: VM >2014-01-09 13:57:13,998 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (DefaultQuartzScheduler_Worker-91) START, IsoPrefixVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 7b56f582 >2014-01-09 13:57:13,999 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (DefaultQuartzScheduler_Worker-91) FINISH, IsoPrefixVDSCommand, return: /rhev/data-center/mnt/192.168.101.101:_iso/de2fccd8-63cc-4662-8878-077b4cfb810e/images/11111111-1111-1111-1111-111111111111, log id: 7b56f582 >2014-01-09 13:57:14,002 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (DefaultQuartzScheduler_Worker-91) START, CreateVmVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, vm=VM [VETS]), log id: 7b891975 >2014-01-09 13:57:14,005 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (DefaultQuartzScheduler_Worker-91) START, CreateVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, vm=VM [VETS]), log id: 720e3dab >2014-01-09 13:57:27,741 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-15) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 13:57:27,743 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-15) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 13:57:27,746 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-15) SPM selection - vds seems as spm ovirt01 >2014-01-09 13:57:27,748 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-15) START, SpmStopVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 749f67f5 >2014-01-09 13:57:34,089 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (DefaultQuartzScheduler_Worker-91) org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand spiceSslCipherSuite=DEFAULT,memSize=384,kvmEnable=true,smp=1,vmType=kvm,emulatedMachine=rhel6.4.0,keyboardLayout=en-us,memGuaranteedSize=384,pitReinjection=false,nice=0,display=qxl,smartcardEnable=false,smpCoresPerSocket=1,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,timeOffset=0,transparentHugePages=true,vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa,devices=[Ljava.util.HashMap;@7b6087b5,acpiEnable=true,vmName=VETS,cpuType=Nehalem,custom={device_3b9fd567-0391-480b-b060-5eab8c8755b2device_8126e458-b89c-4dff-a50b-501196489d9bdevice_e02a1d9f-53fd-4946-94a9-1b355f514b48device_d3e1c40e-1ca6-4670-9a5a-37fc96e4888fdevice_6e3468bc-a9e2-4bf5-a387-60b0427052b9=VmDevice {vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, deviceId=6e3468bc-a9e2-4bf5-a387-60b0427052b9, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x04, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}}, device_3b9fd567-0391-480b-b060-5eab8c8755b2device_8126e458-b89c-4dff-a50b-501196489d9bdevice_e02a1d9f-53fd-4946-94a9-1b355f514b48=VmDevice {vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, deviceId=e02a1d9f-53fd-4946-94a9-1b355f514b48, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}}, device_3b9fd567-0391-480b-b060-5eab8c8755b2device_8126e458-b89c-4dff-a50b-501196489d9b=VmDevice {vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, deviceId=8126e458-b89c-4dff-a50b-501196489d9b, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}}, device_3b9fd567-0391-480b-b060-5eab8c8755b2=VmDevice {vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, deviceId=3b9fd567-0391-480b-b060-5eab8c8755b2, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}}, device_3b9fd567-0391-480b-b060-5eab8c8755b2device_8126e458-b89c-4dff-a50b-501196489d9bdevice_e02a1d9f-53fd-4946-94a9-1b355f514b48device_d3e1c40e-1ca6-4670-9a5a-37fc96e4888f=VmDevice {vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, deviceId=d3e1c40e-1ca6-4670-9a5a-37fc96e4888f, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}}} >2014-01-09 13:57:34,100 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (DefaultQuartzScheduler_Worker-91) FINISH, CreateVDSCommand, log id: 720e3dab >2014-01-09 13:57:34,103 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (DefaultQuartzScheduler_Worker-91) FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 7b891975 >2014-01-09 13:57:34,103 INFO [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-91) Lock freed to object EngineLock [exclusiveLocks= key: 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa value: VM >, sharedLocks= ] >2014-01-09 13:57:34,107 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-91) Correlation ID: ab7e414, Job ID: 4ec53e4b-a8cd-4462-8855-65e063b758c9, Call Stack: null, Custom Event ID: -1, Message: VM VETS was restarted on Host ovirt02 >2014-01-09 13:57:34,113 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-91) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Highly Available VM External_Email failed. It will be restarted automatically. >2014-01-09 13:57:34,114 INFO [org.ovirt.engine.core.bll.VdsEventListener] (DefaultQuartzScheduler_Worker-91) Highly Available VM went down. Attempting to restart. VM Name: External_Email, VM Id:872c3dea-6984-48fa-99c3-354fd367c3f9 >2014-01-09 13:57:34,129 INFO [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-91) Lock Acquired to object EngineLock [exclusiveLocks= key: 872c3dea-6984-48fa-99c3-354fd367c3f9 value: VM >, sharedLocks= ] >2014-01-09 13:57:34,144 WARN [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-91) CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,VM_CANNOT_RUN_FROM_CD_WITHOUT_ACTIVE_STORAGE_DOMAIN_ISO >2014-01-09 13:57:34,145 INFO [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-91) Lock freed to object EngineLock [exclusiveLocks= key: 872c3dea-6984-48fa-99c3-354fd367c3f9 value: VM >, sharedLocks= ] >2014-01-09 13:57:34,150 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-91) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Restart of the Highly Available VM External_Email failed. >2014-01-09 13:57:47,806 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand return value > >TaskStatusListReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=654, mMessage=Not SPM]] > >2014-01-09 13:57:47,808 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-15) HostName = ovirt01 >2014-01-09 13:57:47,808 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command HSMGetAllTasksStatusesVDS execution failed. Exception: IRSNonOperationalException: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM >2014-01-09 13:57:47,809 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-15) FINISH, SpmStopVDSCommand, log id: 749f67f5 >2014-01-09 13:57:47,810 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-15) spm stop succeeded, continuing with spm selection >2014-01-09 13:57:47,812 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-15) starting spm on vds ovirt02, storage pool CSEnt, prevId 1, LVER 11 >2014-01-09 13:57:47,813 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-15) START, SpmStartVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, prevId=1, prevLVER=11, storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=true), log id: 3cd35ecb >2014-01-09 13:58:07,848 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-15) spmStart polling started: taskId = 03ee0670-7efe-4731-a90a-8b362f90fb52 >2014-01-09 13:58:14,174 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) VM VETS 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa moved from WaitForLaunch --> PoweringUp >2014-01-09 13:58:14,175 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-47) START, FullListVdsCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02], vmIds=[60ace091-8a4e-4047-9ac0-c0ea95e3c5fa]), log id: 288b2568 >2014-01-09 13:58:28,878 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-15) spmStart polling ended: taskId = 03ee0670-7efe-4731-a90a-8b362f90fb52 task status = finished >2014-01-09 13:58:34,209 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-47) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@737a14e, log id: 288b2568 >2014-01-09 13:58:34,212 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) Received a memballoon Device without an address when processing VM 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa devices, skipping device: {alias=balloon0, specParams={model=none}, device=memballoon, type=balloon} >2014-01-09 13:58:48,909 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-15) spmStart polling ended, spm status: SPM >2014-01-09 13:58:48,911 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-15) START, HSMClearTaskVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, taskId=03ee0670-7efe-4731-a90a-8b362f90fb52), log id: 167986b7 >2014-01-09 13:59:08,946 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-15) FINISH, HSMClearTaskVDSCommand, log id: 167986b7 >2014-01-09 13:59:08,947 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-15) FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.businessentities.SpmStatusResult@39153069, log id: 3cd35ecb >2014-01-09 13:59:08,949 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-15) Initialize Irs proxy from vds: 192.168.100.22 >2014-01-09 13:59:08,953 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-15) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host ovirt02 (Address: 192.168.100.22). >2014-01-09 13:59:08,963 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-6-thread-47) START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false), log id: 6bd09aaf >2014-01-09 13:59:17,282 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-25) VM VETS 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa moved from PoweringUp --> Up >2014-01-09 13:59:17,286 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-25) Correlation ID: ab7e414, Job ID: 4ec53e4b-a8cd-4462-8855-65e063b758c9, Call Stack: null, Custom Event ID: -1, Message: VM VETS was restarted on Host ovirt02 >2014-01-09 13:59:28,989 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-6-thread-47) -- executeIrsBrokerCommand: Attempting on storage pool 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 13:59:28,991 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (pool-6-thread-47) START, HSMGetAllTasksInfoVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5), log id: 5207e890 >2014-01-09 13:59:49,022 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (pool-6-thread-47) FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 5207e890 >2014-01-09 13:59:49,023 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-6-thread-47) FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 6bd09aaf >2014-01-09 13:59:49,024 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (pool-6-thread-47) Discovered no tasks on Storage Pool CSEnt >2014-01-09 14:11:32,427 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp--127.0.0.1-8702-17) Running command: LoginUserCommand internal: false. >2014-01-09 14:11:32,432 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-17) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2014-01-09 14:12:22,074 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-14) Lock Acquired to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 14:12:22,112 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-14) START, IsVmDuringInitiatingVDSCommand( vmId = b6972001-2bd6-4cea-9c15-c046674754d4), log id: 4e3552ce >2014-01-09 14:12:22,113 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-14) FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 4e3552ce >2014-01-09 14:12:22,130 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-14) Running command: RunVmCommand internal: false. Entities affected : ID: b6972001-2bd6-4cea-9c15-c046674754d4 Type: VM >2014-01-09 14:12:22,148 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (ajp--127.0.0.1-8702-14) START, IsoPrefixVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 7758c210 >2014-01-09 14:12:22,149 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (ajp--127.0.0.1-8702-14) FINISH, IsoPrefixVDSCommand, return: /rhev/data-center/mnt/192.168.101.101:_iso/de2fccd8-63cc-4662-8878-077b4cfb810e/images/11111111-1111-1111-1111-111111111111, log id: 7758c210 >2014-01-09 14:12:22,153 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (ajp--127.0.0.1-8702-14) START, CreateVmVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=b6972001-2bd6-4cea-9c15-c046674754d4, vm=VM [DC-01]), log id: 52b893e >2014-01-09 14:12:25,452 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (ajp--127.0.0.1-8702-14) START, CreateVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=b6972001-2bd6-4cea-9c15-c046674754d4, vm=VM [DC-01]), log id: 603b8894 >2014-01-09 14:12:45,513 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (ajp--127.0.0.1-8702-14) org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand spiceSslCipherSuite=DEFAULT,memSize=2048,kvmEnable=true,smp=2,vmType=kvm,emulatedMachine=rhel6.4.0,keyboardLayout=en-us,memGuaranteedSize=2048,nice=0,display=qxl,smartcardEnable=false,smpCoresPerSocket=1,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,timeOffset=-3,transparentHugePages=true,vmId=b6972001-2bd6-4cea-9c15-c046674754d4,devices=[Ljava.util.HashMap;@305fb274,acpiEnable=true,vmName=DC-01,cpuType=Nehalem,custom={device_1e0ab694-8ef3-4108-9a6a-a12bcfdda845device_53cd3018-54fe-46dc-9a68-4796a2fdb2f6device_5ae289bc-d2ec-42f9-9e04-3c4012c2cf01device_8eb031e6-9a14-4873-bf8d-afc44b87e0cfdevice_cab14d02-9c35-425f-9400-478da2bd13a4=VmDevice {vmId=b6972001-2bd6-4cea-9c15-c046674754d4, deviceId=cab14d02-9c35-425f-9400-478da2bd13a4, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}}, device_1e0ab694-8ef3-4108-9a6a-a12bcfdda845device_53cd3018-54fe-46dc-9a68-4796a2fdb2f6=VmDevice {vmId=b6972001-2bd6-4cea-9c15-c046674754d4, deviceId=53cd3018-54fe-46dc-9a68-4796a2fdb2f6, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}}, device_1e0ab694-8ef3-4108-9a6a-a12bcfdda845device_53cd3018-54fe-46dc-9a68-4796a2fdb2f6device_5ae289bc-d2ec-42f9-9e04-3c4012c2cf01=VmDevice {vmId=b6972001-2bd6-4cea-9c15-c046674754d4, deviceId=5ae289bc-d2ec-42f9-9e04-3c4012c2cf01, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}}, device_1e0ab694-8ef3-4108-9a6a-a12bcfdda845device_53cd3018-54fe-46dc-9a68-4796a2fdb2f6device_5ae289bc-d2ec-42f9-9e04-3c4012c2cf01device_8eb031e6-9a14-4873-bf8d-afc44b87e0cf=VmDevice {vmId=b6972001-2bd6-4cea-9c15-c046674754d4, deviceId=8eb031e6-9a14-4873-bf8d-afc44b87e0cf, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}}, device_1e0ab694-8ef3-4108-9a6a-a12bcfdda845=VmDevice {vmId=b6972001-2bd6-4cea-9c15-c046674754d4, deviceId=1e0ab694-8ef3-4108-9a6a-a12bcfdda845, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}}} >2014-01-09 14:12:45,522 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (ajp--127.0.0.1-8702-14) FINISH, CreateVDSCommand, log id: 603b8894 >2014-01-09 14:12:45,525 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (ajp--127.0.0.1-8702-14) FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 52b893e >2014-01-09 14:12:45,526 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-14) Lock freed to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 14:12:45,530 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-14) Correlation ID: 48b8f249, Job ID: d45cc06d-4bf3-47bb-8d95-773ecb6c54ca, Call Stack: null, Custom Event ID: -1, Message: VM DC-01 was started by admin@internal (Host: ovirt02). >2014-01-09 14:12:52,853 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-12) Lock Acquired to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 14:12:52,869 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-12) START, IsVmDuringInitiatingVDSCommand( vmId = 33f88d50-30eb-45c7-9850-733da17ed7a4), log id: 2edfbdcb >2014-01-09 14:12:52,870 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-12) FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 2edfbdcb >2014-01-09 14:12:52,882 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-12) Running command: RunVmCommand internal: false. Entities affected : ID: 33f88d50-30eb-45c7-9850-733da17ed7a4 Type: VM >2014-01-09 14:12:52,899 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (ajp--127.0.0.1-8702-12) START, IsoPrefixVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 333b8f9b >2014-01-09 14:12:52,900 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (ajp--127.0.0.1-8702-12) FINISH, IsoPrefixVDSCommand, return: /rhev/data-center/mnt/192.168.101.101:_iso/de2fccd8-63cc-4662-8878-077b4cfb810e/images/11111111-1111-1111-1111-111111111111, log id: 333b8f9b >2014-01-09 14:12:52,903 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (ajp--127.0.0.1-8702-12) START, CreateVmVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, vm=VM [DC-02]), log id: 388d0263 >2014-01-09 14:12:57,501 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (ajp--127.0.0.1-8702-12) START, CreateVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, vm=VM [DC-02]), log id: 4622d37c >2014-01-09 14:13:17,630 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (ajp--127.0.0.1-8702-12) org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand spiceSslCipherSuite=DEFAULT,memSize=2048,kvmEnable=true,smp=2,vmType=kvm,emulatedMachine=rhel6.4.0,keyboardLayout=en-us,memGuaranteedSize=2048,nice=0,display=qxl,smartcardEnable=false,smpCoresPerSocket=1,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,timeOffset=-18003,transparentHugePages=true,vmId=33f88d50-30eb-45c7-9850-733da17ed7a4,devices=[Ljava.util.HashMap;@4e406ab8,acpiEnable=true,vmName=DC-02,cpuType=Nehalem,custom={device_d2413266-11d3-4f4c-8d54-15bc7125ce7d=VmDevice {vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, deviceId=d2413266-11d3-4f4c-8d54-15bc7125ce7d, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}}, device_d2413266-11d3-4f4c-8d54-15bc7125ce7ddevice_dbeb9626-57de-460a-a944-b6ab9d1a1c39=VmDevice {vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, deviceId=dbeb9626-57de-460a-a944-b6ab9d1a1c39, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}}, device_d2413266-11d3-4f4c-8d54-15bc7125ce7ddevice_dbeb9626-57de-460a-a944-b6ab9d1a1c39device_230cd5ff-7923-464b-880b-4748eaa53b71device_4d2d205b-91f4-4454-853b-0b259eca185f=VmDevice {vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, deviceId=4d2d205b-91f4-4454-853b-0b259eca185f, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}}, device_d2413266-11d3-4f4c-8d54-15bc7125ce7ddevice_dbeb9626-57de-460a-a944-b6ab9d1a1c39device_230cd5ff-7923-464b-880b-4748eaa53b71device_4d2d205b-91f4-4454-853b-0b259eca185fdevice_a8e4b107-61c5-43e6-a3dd-18a918db1957=VmDevice {vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, deviceId=a8e4b107-61c5-43e6-a3dd-18a918db1957, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x04, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}}, device_d2413266-11d3-4f4c-8d54-15bc7125ce7ddevice_dbeb9626-57de-460a-a944-b6ab9d1a1c39device_230cd5ff-7923-464b-880b-4748eaa53b71=VmDevice {vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, deviceId=230cd5ff-7923-464b-880b-4748eaa53b71, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}}} >2014-01-09 14:13:17,641 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (ajp--127.0.0.1-8702-12) FINISH, CreateVDSCommand, log id: 4622d37c >2014-01-09 14:13:17,644 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (ajp--127.0.0.1-8702-12) FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 388d0263 >2014-01-09 14:13:17,644 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-12) Lock freed to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 14:13:17,648 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-12) Correlation ID: 7cfee70c, Job ID: 5bad6064-bc21-4708-8801-f67013cfa1db, Call Stack: null, Custom Event ID: -1, Message: VM DC-02 was started by admin@internal (Host: ovirt01). >2014-01-09 14:13:25,584 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) VM DC-01 b6972001-2bd6-4cea-9c15-c046674754d4 moved from WaitForLaunch --> PoweringUp >2014-01-09 14:13:25,585 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-80) START, FullListVdsCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02], vmIds=[b6972001-2bd6-4cea-9c15-c046674754d4]), log id: 372e1e5b >2014-01-09 14:13:45,619 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-80) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@40ae43b7, log id: 372e1e5b >2014-01-09 14:13:52,705 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-41) VM DC-02 33f88d50-30eb-45c7-9850-733da17ed7a4 moved from WaitForLaunch --> PoweringUp >2014-01-09 14:13:52,706 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-41) START, FullListVdsCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01], vmIds=[33f88d50-30eb-45c7-9850-733da17ed7a4]), log id: 538f4dd4 >2014-01-09 14:14:01,908 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp--127.0.0.1-8702-14) Running command: LoginUserCommand internal: false. >2014-01-09 14:14:01,912 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-14) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2014-01-09 14:14:07,720 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-41) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@3f514b05, log id: 538f4dd4 >2014-01-09 14:14:18,684 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) VM DC-01 b6972001-2bd6-4cea-9c15-c046674754d4 moved from PoweringUp --> Up >2014-01-09 14:14:18,689 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-82) Correlation ID: 48b8f249, Job ID: d45cc06d-4bf3-47bb-8d95-773ecb6c54ca, Call Stack: null, Custom Event ID: -1, Message: VM DC-01 started on Host ovirt02 >2014-01-09 14:14:45,788 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-92) VM DC-02 33f88d50-30eb-45c7-9850-733da17ed7a4 moved from PoweringUp --> Up >2014-01-09 14:14:45,792 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-92) Correlation ID: 7cfee70c, Job ID: 5bad6064-bc21-4708-8801-f67013cfa1db, Call Stack: null, Custom Event ID: -1, Message: VM DC-02 started on Host ovirt01 >2014-01-09 14:20:21,047 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp--127.0.0.1-8702-11) Running command: SetVmTicketCommand internal: false. Entities affected : ID: b6972001-2bd6-4cea-9c15-c046674754d4 Type: VM >2014-01-09 14:20:21,052 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp--127.0.0.1-8702-11) START, SetVmTicketVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=b6972001-2bd6-4cea-9c15-c046674754d4, ticket=pwOP0BGtCOmp, validTime=120,m userName=admin@internal, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 73c95bd4 >2014-01-09 14:20:21,096 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp--127.0.0.1-8702-11) FINISH, SetVmTicketVDSCommand, log id: 73c95bd4 >2014-01-09 14:20:21,108 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-11) Correlation ID: 67306ba8, Call Stack: null, Custom Event ID: -1, Message: user admin@internal initiated console session for VM DC-01 >2014-01-09 14:20:53,817 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-20) [1de28692] Lock Acquired to object EngineLock [exclusiveLocks= key: 872c3dea-6984-48fa-99c3-354fd367c3f9 value: VM >, sharedLocks= ] >2014-01-09 14:20:53,833 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-20) [1de28692] START, IsVmDuringInitiatingVDSCommand( vmId = 872c3dea-6984-48fa-99c3-354fd367c3f9), log id: 2530a9b5 >2014-01-09 14:20:53,834 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-20) [1de28692] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 2530a9b5 >2014-01-09 14:20:53,853 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-6-thread-50) [1de28692] Running command: RunVmCommand internal: false. Entities affected : ID: 872c3dea-6984-48fa-99c3-354fd367c3f9 Type: VM >2014-01-09 14:20:53,870 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-6-thread-50) [1de28692] START, IsoPrefixVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 67c2728c >2014-01-09 14:20:53,871 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-6-thread-50) [1de28692] FINISH, IsoPrefixVDSCommand, return: /rhev/data-center/mnt/192.168.101.101:_iso/de2fccd8-63cc-4662-8878-077b4cfb810e/images/11111111-1111-1111-1111-111111111111, log id: 67c2728c >2014-01-09 14:20:53,875 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-6-thread-50) [1de28692] START, CreateVmVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, vm=VM [External_Email]), log id: 2ce11c64 >2014-01-09 14:20:53,877 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-50) [1de28692] START, CreateVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, vm=VM [External_Email]), log id: 5a29e467 >2014-01-09 14:20:53,950 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-50) [1de28692] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand spiceSslCipherSuite=DEFAULT,memSize=2048,kvmEnable=true,smp=2,vmType=kvm,emulatedMachine=rhel6.4.0,keyboardLayout=en-us,memGuaranteedSize=2048,pitReinjection=false,nice=0,display=qxl,smartcardEnable=false,smpCoresPerSocket=1,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,timeOffset=-2,transparentHugePages=true,vmId=872c3dea-6984-48fa-99c3-354fd367c3f9,devices=[Ljava.util.HashMap;@55dd0de8,acpiEnable=true,vmName=External_Email,cpuType=Nehalem,custom={device_564f5a24-3d22-4553-93c3-efe71fff9167device_a4639197-8804-4bcf-83cc-f8f21658f40f=VmDevice {vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, deviceId=a4639197-8804-4bcf-83cc-f8f21658f40f, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}}, device_564f5a24-3d22-4553-93c3-efe71fff9167device_a4639197-8804-4bcf-83cc-f8f21658f40fdevice_0a862e1b-c7da-4e7a-9e49-a4471389540cdevice_02bc08b3-9e6e-4b3e-9e62-99a8ca6f9593=VmDevice {vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, deviceId=02bc08b3-9e6e-4b3e-9e62-99a8ca6f9593, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}}, device_564f5a24-3d22-4553-93c3-efe71fff9167device_a4639197-8804-4bcf-83cc-f8f21658f40fdevice_0a862e1b-c7da-4e7a-9e49-a4471389540c=VmDevice {vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, deviceId=0a862e1b-c7da-4e7a-9e49-a4471389540c, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}}, device_564f5a24-3d22-4553-93c3-efe71fff9167=VmDevice {vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, deviceId=564f5a24-3d22-4553-93c3-efe71fff9167, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}}, device_564f5a24-3d22-4553-93c3-efe71fff9167device_a4639197-8804-4bcf-83cc-f8f21658f40fdevice_0a862e1b-c7da-4e7a-9e49-a4471389540cdevice_02bc08b3-9e6e-4b3e-9e62-99a8ca6f9593device_9c838b27-29f2-4cf5-8813-3dd0952809f0=VmDevice {vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, deviceId=9c838b27-29f2-4cf5-8813-3dd0952809f0, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x04, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}}} >2014-01-09 14:20:53,957 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-50) [1de28692] FINISH, CreateVDSCommand, log id: 5a29e467 >2014-01-09 14:20:53,959 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-6-thread-50) [1de28692] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 2ce11c64 >2014-01-09 14:20:53,960 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-6-thread-50) [1de28692] Lock freed to object EngineLock [exclusiveLocks= key: 872c3dea-6984-48fa-99c3-354fd367c3f9 value: VM >, sharedLocks= ] >2014-01-09 14:20:53,962 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-50) [1de28692] Correlation ID: 1de28692, Job ID: 0ca18562-62af-46db-b558-8f00065f6c1b, Call Stack: null, Custom Event ID: -1, Message: VM External_Email was started by admin@internal (Host: ovirt01). >2014-01-09 14:20:57,774 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) VM External_Email 872c3dea-6984-48fa-99c3-354fd367c3f9 moved from WaitForLaunch --> PoweringUp >2014-01-09 14:20:57,778 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-2) START, FullListVdsCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01], vmIds=[872c3dea-6984-48fa-99c3-354fd367c3f9]), log id: 644f2f0d >2014-01-09 14:20:57,789 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-2) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@114405b2, log id: 644f2f0d >2014-01-09 14:21:55,302 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-69) VM External_Email 872c3dea-6984-48fa-99c3-354fd367c3f9 moved from PoweringUp --> Up >2014-01-09 14:21:55,306 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-69) Correlation ID: 1de28692, Job ID: 0ca18562-62af-46db-b558-8f00065f6c1b, Call Stack: null, Custom Event ID: -1, Message: VM External_Email started on Host ovirt01 >2014-01-09 14:23:46,704 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp--127.0.0.1-8702-9) Running command: SetVmTicketCommand internal: false. Entities affected : ID: 872c3dea-6984-48fa-99c3-354fd367c3f9 Type: VM >2014-01-09 14:23:46,708 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp--127.0.0.1-8702-9) START, SetVmTicketVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, ticket=Y0pw1euuCPeL, validTime=120,m userName=admin@internal, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 1234d895 >2014-01-09 14:23:46,748 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp--127.0.0.1-8702-9) FINISH, SetVmTicketVDSCommand, log id: 1234d895 >2014-01-09 14:23:46,753 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-9) Correlation ID: 19b87251, Call Stack: null, Custom Event ID: -1, Message: user admin@internal initiated console session for VM External_Email >2014-01-09 14:24:39,730 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-16) Lock Acquired to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 14:24:39,738 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-16) CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,ACTION_TYPE_FAILED_VM_STATUS_ILLEGAL,VAR__VM_STATUS__UNKNOWN >2014-01-09 14:24:39,738 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-16) Lock freed to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 14:24:39,790 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp--127.0.0.1-8702-16) Operation Failed: [Cannot run VM because it is in Unknown status.] >2014-01-09 14:25:12,642 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-60) Attempting to update VMs/Templates Ovf. >2014-01-09 14:25:12,644 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-60) Attempting to update VM OVFs in Data Center CSEnt >2014-01-09 14:25:12,645 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-60) Successfully updated VM OVFs in Data Center CSEnt >2014-01-09 14:25:12,646 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-60) Attempting to update template OVFs in Data Center CSEnt >2014-01-09 14:25:12,647 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-60) Successfully updated templates OVFs in Data Center CSEnt >2014-01-09 14:25:12,647 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-60) Attempting to remove unneeded template/vm OVFs in Data Center CSEnt >2014-01-09 14:25:12,648 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-60) Successfully removed unneeded template/vm OVFs in Data Center CSEnt >2014-01-09 14:26:43,278 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-48) Start refreshing all users data >2014-01-09 14:26:43,458 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-48) Inactive User Patrick found in directory server, its status switched to Active >2014-01-09 14:26:43,461 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-48) Inactive User Chris found in directory server, its status switched to Active >2014-01-09 14:26:43,463 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-48) Inactive User Patrick found in directory server, its status switched to Active >2014-01-09 14:26:43,464 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-48) Inactive User Ryan found in directory server, its status switched to Active >2014-01-09 14:26:43,465 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-48) Inactive User Ryan found in directory server, its status switched to Active >2014-01-09 14:30:17,937 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetIsoListVDSCommand] (pool-6-thread-50) START, GetIsoListVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false), log id: 7f7f98cd >2014-01-09 14:30:17,939 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HsmGetIsoListVDSCommand] (pool-6-thread-50) START, HsmGetIsoListVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 1208fc20 >2014-01-09 14:30:17,991 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HsmGetIsoListVDSCommand] (pool-6-thread-50) FINISH, HsmGetIsoListVDSCommand, return: [virtio-win-0.1-52.iso, centos64x64netinst.iso, virtio-win-0.1-65.iso, winserve2012.iso, kali-linux-1.0.5-amd641.iso, Server2008R2SP1-en-US-July2013.iso, win7.iso, virt-p2v-0.9.1-2.20130730.1.el6.centos.iso, zentyal-3.2-amd64.iso, Elastix-2.4.0-Stable-x86_64-bin-04feb2013.iso, EXCH201064.ISO], log id: 1208fc20 >2014-01-09 14:30:17,993 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetIsoListVDSCommand] (pool-6-thread-50) FINISH, GetIsoListVDSCommand, return: [virtio-win-0.1-52.iso, centos64x64netinst.iso, virtio-win-0.1-65.iso, winserve2012.iso, kali-linux-1.0.5-amd641.iso, Server2008R2SP1-en-US-July2013.iso, win7.iso, virt-p2v-0.9.1-2.20130730.1.el6.centos.iso, zentyal-3.2-amd64.iso, Elastix-2.4.0-Stable-x86_64-bin-04feb2013.iso, EXCH201064.ISO], log id: 7f7f98cd >2014-01-09 14:30:17,998 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (pool-6-thread-50) Finished automatic refresh process for ISO file type with success, for storage domain id de2fccd8-63cc-4662-8878-077b4cfb810e. >2014-01-09 14:30:57,359 INFO [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-6) Running command: LoginAdminUserCommand internal: false. >2014-01-09 14:30:57,364 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User patrick-a logged in. >2014-01-09 14:30:57,555 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp--127.0.0.1-8702-4) Running command: LoginUserCommand internal: false. >2014-01-09 14:30:57,561 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User patrick-a logged in. >2014-01-09 14:36:28,384 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp--127.0.0.1-8702-9) Lock Acquired to object EngineLock [exclusiveLocks= key: syslog00 value: VM_NAME >, sharedLocks= ] >2014-01-09 14:36:28,394 WARN [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp--127.0.0.1-8702-9) CanDoAction of action UpdateVm failed. Reasons:VAR__ACTION__UPDATE,VAR__TYPE__VM,$device VirtIO-SCSI,VM_CANNOT_UPDATE_DEVICE_VM_NOT_DOWN >2014-01-09 14:36:28,395 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp--127.0.0.1-8702-9) Lock freed to object EngineLock [exclusiveLocks= key: syslog00 value: VM_NAME >, sharedLocks= ] >2014-01-09 14:43:08,075 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (ajp--127.0.0.1-8702-19) [415d7cba] Lock Acquired to object EngineLock [exclusiveLocks= key: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 value: VDS >, sharedLocks= ] >2014-01-09 14:43:08,079 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (pool-6-thread-48) [415d7cba] Running command: RefreshHostCapabilitiesCommand internal: false. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 14:43:08,081 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (pool-6-thread-48) [415d7cba] Before acquiring lock in order to prevent monitoring for host ovirt01 from data-center CSEnt >2014-01-09 14:43:08,082 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (pool-6-thread-48) [415d7cba] Lock acquired, from now a monitoring of host will be skipped for host ovirt01 from data-center CSEnt >2014-01-09 14:43:08,169 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (pool-6-thread-48) [415d7cba] START, GetHardwareInfoVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01]), log id: 37d1e2b4 >2014-01-09 14:43:08,191 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (pool-6-thread-48) [415d7cba] FINISH, GetHardwareInfoVDSCommand, log id: 37d1e2b4 >2014-01-09 14:43:08,203 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (pool-6-thread-48) Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 14:43:08,206 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (pool-6-thread-48) Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 14:43:08,208 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (pool-6-thread-48) Refresh host capabilities finished. Lock released. Monitoring can run now for host ovirt01 from data-center CSEnt >2014-01-09 14:43:08,212 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) Correlation ID: 415d7cba, Call Stack: null, Custom Event ID: -1, Message: Successfully refreshed the capabilities of host ovirt01. >2014-01-09 14:43:08,214 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (pool-6-thread-48) Lock freed to object EngineLock [exclusiveLocks= key: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 value: VDS >, sharedLocks= ] >2014-01-09 14:43:20,536 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (ajp--127.0.0.1-8702-19) [47f32f3e] Lock Acquired to object EngineLock [exclusiveLocks= key: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 value: VDS >, sharedLocks= ] >2014-01-09 14:43:20,539 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (pool-6-thread-48) [47f32f3e] Running command: RefreshHostCapabilitiesCommand internal: false. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 14:43:20,540 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (pool-6-thread-48) [47f32f3e] Before acquiring lock in order to prevent monitoring for host ovirt02 from data-center CSEnt >2014-01-09 14:43:20,543 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (pool-6-thread-48) [47f32f3e] Lock acquired, from now a monitoring of host will be skipped for host ovirt02 from data-center CSEnt >2014-01-09 14:43:20,639 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (pool-6-thread-48) [47f32f3e] START, GetHardwareInfoVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02]), log id: 404bc0eb >2014-01-09 14:43:20,661 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (pool-6-thread-48) [47f32f3e] FINISH, GetHardwareInfoVDSCommand, log id: 404bc0eb >2014-01-09 14:43:20,672 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (pool-6-thread-48) Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 14:43:20,675 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (pool-6-thread-48) Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 14:43:20,676 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (pool-6-thread-48) Refresh host capabilities finished. Lock released. Monitoring can run now for host ovirt02 from data-center CSEnt >2014-01-09 14:43:20,679 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) Correlation ID: 47f32f3e, Call Stack: null, Custom Event ID: -1, Message: Successfully refreshed the capabilities of host ovirt02. >2014-01-09 14:43:20,681 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (pool-6-thread-48) Lock freed to object EngineLock [exclusiveLocks= key: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 value: VDS >, sharedLocks= ] >2014-01-09 15:04:59,877 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp--127.0.0.1-8702-11) Running command: LoginUserCommand internal: false. >2014-01-09 15:04:59,883 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2014-01-09 15:05:15,282 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-4) Lock Acquired to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 15:05:15,290 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-4) CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,ACTION_TYPE_FAILED_VM_STATUS_ILLEGAL,VAR__VM_STATUS__UNKNOWN >2014-01-09 15:05:15,291 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-4) Lock freed to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 15:05:15,295 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp--127.0.0.1-8702-4) Operation Failed: [Cannot run VM because it is in Unknown status.] >2014-01-09 15:25:12,259 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp--127.0.0.1-8702-7) Running command: LogoutUserCommand internal: false. >2014-01-09 15:25:12,263 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-7) Correlation ID: 5c7dd27a, Call Stack: null, Custom Event ID: -1, Message: User patrick-a@cytechservices.local logged out. >2014-01-09 15:25:12,649 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-27) Attempting to update VMs/Templates Ovf. >2014-01-09 15:25:12,651 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-27) Attempting to update VM OVFs in Data Center CSEnt >2014-01-09 15:25:12,660 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-27) Successfully updated VM OVFs in Data Center CSEnt >2014-01-09 15:25:12,661 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-27) Attempting to update template OVFs in Data Center CSEnt >2014-01-09 15:25:12,663 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-27) Successfully updated templates OVFs in Data Center CSEnt >2014-01-09 15:25:12,664 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-27) Attempting to remove unneeded template/vm OVFs in Data Center CSEnt >2014-01-09 15:25:12,666 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-27) Successfully removed unneeded template/vm OVFs in Data Center CSEnt >2014-01-09 15:26:43,467 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-13) Start refreshing all users data >2014-01-09 15:28:45,723 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp--127.0.0.1-8702-8) Running command: LoginUserCommand internal: false. >2014-01-09 15:28:45,727 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2014-01-09 15:30:18,001 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetIsoListVDSCommand] (pool-6-thread-46) START, GetIsoListVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false), log id: 74eca4a4 >2014-01-09 15:30:18,003 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HsmGetIsoListVDSCommand] (pool-6-thread-46) START, HsmGetIsoListVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 5c2d46f4 >2014-01-09 15:30:18,038 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HsmGetIsoListVDSCommand] (pool-6-thread-46) FINISH, HsmGetIsoListVDSCommand, return: [virtio-win-0.1-52.iso, centos64x64netinst.iso, virtio-win-0.1-65.iso, winserve2012.iso, kali-linux-1.0.5-amd641.iso, Server2008R2SP1-en-US-July2013.iso, win7.iso, virt-p2v-0.9.1-2.20130730.1.el6.centos.iso, zentyal-3.2-amd64.iso, Elastix-2.4.0-Stable-x86_64-bin-04feb2013.iso, EXCH201064.ISO], log id: 5c2d46f4 >2014-01-09 15:30:18,040 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetIsoListVDSCommand] (pool-6-thread-46) FINISH, GetIsoListVDSCommand, return: [virtio-win-0.1-52.iso, centos64x64netinst.iso, virtio-win-0.1-65.iso, winserve2012.iso, kali-linux-1.0.5-amd641.iso, Server2008R2SP1-en-US-July2013.iso, win7.iso, virt-p2v-0.9.1-2.20130730.1.el6.centos.iso, zentyal-3.2-amd64.iso, Elastix-2.4.0-Stable-x86_64-bin-04feb2013.iso, EXCH201064.ISO], log id: 74eca4a4 >2014-01-09 15:30:18,045 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (pool-6-thread-46) Finished automatic refresh process for ISO file type with success, for storage domain id de2fccd8-63cc-4662-8878-077b4cfb810e. >2014-01-09 15:31:33,408 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp--127.0.0.1-8702-5) Lock Acquired to object EngineLock [exclusiveLocks= key: syslog00 value: VM_NAME >, sharedLocks= ] >2014-01-09 15:31:33,412 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp--127.0.0.1-8702-5) Running command: UpdateVmCommand internal: false. Entities affected : ID: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 Type: VM >2014-01-09 15:31:33,433 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-5) Correlation ID: 3287b3d9, Job ID: d0699d48-5242-4ff9-a7da-105b23565a0a, Call Stack: null, Custom Event ID: -1, Message: VM syslog00 configuration was updated by admin@internal. >2014-01-09 15:31:33,436 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp--127.0.0.1-8702-5) Lock freed to object EngineLock [exclusiveLocks= key: syslog00 value: VM_NAME >, sharedLocks= ] >2014-01-09 15:31:43,289 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp--127.0.0.1-8702-15) Lock Acquired to object EngineLock [exclusiveLocks= key: syslog00 value: VM_NAME >, sharedLocks= ] >2014-01-09 15:31:43,293 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp--127.0.0.1-8702-15) Running command: UpdateVmCommand internal: false. Entities affected : ID: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 Type: VM >2014-01-09 15:31:43,308 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-15) Correlation ID: 5967f0bd, Job ID: c6389271-9bd7-4352-87d3-d9fa8c474ded, Call Stack: null, Custom Event ID: -1, Message: VM syslog00 configuration was updated by admin@internal. >2014-01-09 15:31:43,310 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp--127.0.0.1-8702-15) Lock freed to object EngineLock [exclusiveLocks= key: syslog00 value: VM_NAME >, sharedLocks= ] >2014-01-09 15:32:01,671 WARN [org.ovirt.engine.core.bll.StopVmCommand] (ajp--127.0.0.1-8702-14) CanDoAction of action StopVm failed. Reasons:VAR__ACTION__STOP,VAR__TYPE__VM,ACTION_TYPE_FAILED_VM_IS_NOT_RUNNING >2014-01-09 15:32:01,674 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp--127.0.0.1-8702-14) Operation Failed: [Cannot stop VM. VM is not running.] >2014-01-09 15:32:11,097 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-6) Lock Acquired to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 15:32:11,105 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-6) CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,ACTION_TYPE_FAILED_VM_STATUS_ILLEGAL,VAR__VM_STATUS__UNKNOWN >2014-01-09 15:32:11,106 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-6) Lock freed to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 15:32:11,110 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp--127.0.0.1-8702-6) Operation Failed: [Cannot run VM because it is in Unknown status.] >2014-01-09 15:40:43,233 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-8) Lock Acquired to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 15:40:43,239 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-8) CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,ACTION_TYPE_FAILED_VM_STATUS_ILLEGAL,VAR__VM_STATUS__UNKNOWN >2014-01-09 15:40:43,239 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-8) Lock freed to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 15:40:43,254 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp--127.0.0.1-8702-8) Operation Failed: [Cannot run VM because it is in Unknown status.] >2014-01-09 15:46:24,350 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-01-09 15:46:24,380 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Loaded file "/etc/ovirt-engine/engine.conf". >2014-01-09 15:46:24,381 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-01-09 15:46:24,382 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-java.conf". >2014-01-09 15:46:24,382 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-01-09 15:46:24,383 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-01-09 15:46:24,384 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-01-09 15:46:24,385 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_AJP_ENABLED" is "true". >2014-01-09 15:46:24,385 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_AJP_PORT" is "8702". >2014-01-09 15:46:24,386 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_APPS" is "engine.ear". >2014-01-09 15:46:24,386 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine". >2014-01-09 15:46:24,387 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-01-09 15:46:24,387 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-01-09 15:46:24,388 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-01-09 15:46:24,388 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-01-09 15:46:24,389 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_DB_HOST" is "localhost". >2014-01-09 15:46:24,389 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-01-09 15:46:24,390 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-01-09 15:46:24,391 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-01-09 15:46:24,391 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_DB_PORT" is "5432". >2014-01-09 15:46:24,392 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_DB_SECURED" is "False". >2014-01-09 15:46:24,392 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-01-09 15:46:24,392 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_DB_SSL" is "false". >2014-01-09 15:46:24,394 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?&sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-01-09 15:46:24,395 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_DB_USER" is "engine". >2014-01-09 15:46:24,395 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_DEBUG_ADDRESS" is "". >2014-01-09 15:46:24,395 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine". >2014-01-09 15:46:24,396 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_ETC" is "/etc/ovirt-engine". >2014-01-09 15:46:24,396 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_FQDN" is "ovirt00.mgmt.cytechservices.com". >2014-01-09 15:46:24,397 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_GROUP" is "ovirt". >2014-01-09 15:46:24,397 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-01-09 15:46:24,397 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-01-09 15:46:24,398 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_HTTPS_ENABLED" is "false". >2014-01-09 15:46:24,398 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_HTTPS_PORT" is "None". >2014-01-09 15:46:24,398 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-01-09 15:46:24,399 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_HTTP_ENABLED" is "false". >2014-01-09 15:46:24,399 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_HTTP_PORT" is "None". >2014-01-09 15:46:24,399 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"". >2014-01-09 15:46:24,400 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine". >2014-01-09 15:46:24,400 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_LOG_TO_CONSOLE" is "false". >2014-01-09 15:46:24,400 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_MANUAL" is "/usr/share/ovirt-engine/manual". >2014-01-09 15:46:24,401 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_PERM_MAX" is "256m". >2014-01-09 15:46:24,401 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_PERM_MIN" is "256m". >2014-01-09 15:46:24,401 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine". >2014-01-09 15:46:24,402 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem". >2014-01-09 15:46:24,402 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer". >2014-01-09 15:46:24,403 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12". >2014-01-09 15:46:24,403 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-01-09 15:46:24,403 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-01-09 15:46:24,404 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore". >2014-01-09 15:46:24,404 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-01-09 15:46:24,405 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false". >2014-01-09 15:46:24,405 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_PROXY_ENABLED" is "true". >2014-01-09 15:46:24,405 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-01-09 15:46:24,406 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-01-09 15:46:24,406 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-01-09 15:46:24,406 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_STOP_TIME" is "10". >2014-01-09 15:46:24,407 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine". >2014-01-09 15:46:24,407 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up". >2014-01-09 15:46:24,407 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_USER" is "ovirt". >2014-01-09 15:46:24,408 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine". >2014-01-09 15:46:24,408 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine". >2014-01-09 15:46:24,408 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-01-09 15:46:24,409 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "JAVA_HOME" is "/usr/lib/jvm/jre-1.7.0-openjdk.x86_64". >2014-01-09 15:46:24,409 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "JBOSS_HOME" is "/usr/share/jboss-as". >2014-01-09 15:46:24,409 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-8) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-01-09 15:46:25,303 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Start initializing Backend >2014-01-09 15:46:26,415 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Running ovirt-engine 3.3.1-2.el6 >2014-01-09 15:46:26,417 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-1) Start initializing dictionaries >2014-01-09 15:46:26,420 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-1) Finished initializing dictionaries >2014-01-09 15:46:26,421 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Start initializing AuditLogCleanupManager >2014-01-09 15:46:26,422 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-01-09 15:46:26,564 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-1) Finished initializing AuditLogCleanupManager >2014-01-09 15:46:26,567 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-1) Start initializing TagsDirector >2014-01-09 15:46:26,577 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-1) Tag root added to tree >2014-01-09 15:46:26,683 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-1) Finished initializing TagsDirector >2014-01-09 15:46:26,686 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-1) Start initializing IsoDomainListSyncronizer >2014-01-09 15:46:26,691 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-1) Finished initializing IsoDomainListSyncronizer >2014-01-09 15:46:26,821 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (MSC service thread 1-1) Loaded file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-01-09 15:46:27,054 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Completed initializing handlers >2014-01-09 15:46:27,059 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Start initializing ErrorTranslatorImpl >2014-01-09 15:46:27,079 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-01-09 15:46:27,080 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Finished initializing ErrorTranslatorImpl >2014-01-09 15:46:27,080 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Start initializing ErrorTranslatorImpl >2014-01-09 15:46:27,081 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-1) Finished initializing ErrorTranslatorImpl >2014-01-09 15:46:27,081 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-1) Mark incomplete jobs as UNKNOWN >2014-01-09 15:46:27,144 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-1) Start initializing JobRepositoryCleanupManager >2014-01-09 15:46:27,145 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-1) Finished initializing JobRepositoryCleanupManager >2014-01-09 15:46:27,146 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-1) Start initializing AutoRecoveryManager >2014-01-09 15:46:27,147 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-1) Finished initializing AutoRecoveryManager >2014-01-09 15:46:27,149 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-1) Start initializing ExecutionMessageDirector >2014-01-09 15:46:27,152 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-1) Finished initializing ExecutionMessageDirector >2014-01-09 15:46:27,245 INFO [org.ovirt.engine.core.bll.adbroker.UsersDomainsCacheManagerService] (MSC service thread 1-5) Start initializing UsersDomainsCacheManagerService >2014-01-09 15:46:27,293 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (MSC service thread 1-1) Initialization of AsyncTaskManager completed successfully. >2014-01-09 15:46:27,296 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-1) Start initializing ResourceManager >2014-01-09 15:46:27,379 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (MSC service thread 1-5) Start initializing DbUserCacheManager >2014-01-09 15:46:27,382 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (MSC service thread 1-5) Finished initializing DbUserCacheManager >2014-01-09 15:46:27,382 INFO [org.ovirt.engine.core.bll.adbroker.UsersDomainsCacheManagerService] (MSC service thread 1-5) Finished initializing UsersDomainsCacheManagerService >2014-01-09 15:46:27,583 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-1) Start refreshing all users data >2014-01-09 15:46:27,840 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-1) Entered VdsManager constructor >2014-01-09 15:46:27,919 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-1) Initialize vdsBroker (192.168.100.21,54,321) >2014-01-09 15:46:28,301 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-1) VDS f2d7a8a9-e77e-4e9a-8031-23a0731407c8 was added to the Resource Manager >2014-01-09 15:46:28,302 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-1) Entered VdsManager constructor >2014-01-09 15:46:28,433 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-1) Initialize vdsBroker (192.168.100.22,54,321) >2014-01-09 15:46:28,458 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-1) VDS bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 was added to the Resource Manager >2014-01-09 15:46:28,589 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-1) Finished initializing ResourceManager >2014-01-09 15:46:28,609 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (MSC service thread 1-1) Initialization of OvfDataUpdater completed successfully. >2014-01-09 15:46:28,613 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-1) Start scheduling to enable vds load balancer >2014-01-09 15:46:28,614 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-1) Finished scheduling to enable vds load balancer >2014-01-09 15:46:28,650 INFO [org.ovirt.engine.core.bll.network.MacPoolManager] (pool-6-thread-1) MacPoolManager(728a5ac2): Start initializing >2014-01-09 15:46:28,657 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-1) Init VM custom properties utilities >2014-01-09 15:46:28,658 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-1) Init device custom properties utilities >2014-01-09 15:46:28,661 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-1) Initializing Scheduling manager >2014-01-09 15:46:28,682 INFO [org.ovirt.engine.core.bll.network.MacPoolManager] (pool-6-thread-1) MacPoolManager(728a5ac2): Finished initializing >2014-01-09 15:46:28,715 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-1) External scheduler disabled, discovery skipped >2014-01-09 15:46:28,716 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-1) Initialized Scheduling manager >2014-01-09 15:46:28,717 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-1) Initializing DWH Heart Beat >2014-01-09 15:46:29,525 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp--127.0.0.1-8702-3) Running command: LogoutUserCommand internal: false. >2014-01-09 15:46:29,580 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) AuditLogType: UNASSIGNED not exist in string table >2014-01-09 15:46:29,581 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-01-09 15:46:29,582 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-01-09 15:46:29,582 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-01-09 15:46:29,583 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-01-09 15:46:29,583 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) AuditLogType: DWH_STOPPED not exist in string table >2014-01-09 15:46:29,584 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) AuditLogType: DWH_STARTED not exist in string table >2014-01-09 15:46:29,584 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) AuditLogType: DWH_ERROR not exist in string table >2014-01-09 15:46:29,591 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) AuditLogType: USER_FAILED_REMOVE_VM not have severity. Assumed Normal >2014-01-09 15:46:29,592 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) AuditLogType: USER_ATTACH_DISK_TO_VM not have severity. Assumed Normal >2014-01-09 15:46:29,592 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) AuditLogType: USER_DETACH_DISK_FROM_VM not have severity. Assumed Normal >2014-01-09 15:46:29,593 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) AuditLogType: USER_FAILED_DETACH_DISK_FROM_VM not have severity. Assumed Normal >2014-01-09 15:46:29,593 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not have severity. Assumed Normal >2014-01-09 15:46:29,594 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) AuditLogType: GLUSTER_VOLUME_OPTION_CHANGED_FROM_CLI not have severity. Assumed Normal >2014-01-09 15:46:29,594 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) AuditLogType: GLUSTER_SERVICES_LIST_NOT_FETCHED not have severity. Assumed Normal >2014-01-09 15:46:29,595 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) AuditLogType: GLUSTER_VOLUME_BRICK_ADDED not have severity. Assumed Normal >2014-01-09 15:46:29,595 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) AuditLogType: USER_EXTEND_DISK_SIZE_UPDATE_VM_FAILURE not have severity. Assumed Normal >2014-01-09 15:46:29,596 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) AuditLogType: VM_MIGRATION_START_SYSTEM_INITIATED not have severity. Assumed Normal >2014-01-09 15:46:29,596 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not have severity. Assumed Normal >2014-01-09 15:46:29,597 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) AuditLogType: NETWORK_UPDATE_VM_INTERFACE_LINK_UP not have severity. Assumed Normal >2014-01-09 15:46:29,597 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) AuditLogType: NETWORK_UPDATE_VM_INTERFACE_LINK_DOWN not have severity. Assumed Normal >2014-01-09 15:46:29,598 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) AuditLogType: USER_UPDATED_MOM_POLICIES not have severity. Assumed Normal >2014-01-09 15:46:29,598 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) AuditLogType: USER_FAILED_TO_UPDATE_MOM_POLICIES not have severity. Assumed Normal >2014-01-09 15:46:29,892 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) Correlation ID: 674522e9, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2014-01-09 15:46:31,905 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-3) START, GetHardwareInfoVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01]), log id: 78f541f2 >2014-01-09 15:46:31,907 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-4) START, GetHardwareInfoVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02]), log id: 2dbcb4e8 >2014-01-09 15:46:31,938 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-4) FINISH, GetHardwareInfoVDSCommand, log id: 2dbcb4e8 >2014-01-09 15:46:31,941 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-3) FINISH, GetHardwareInfoVDSCommand, log id: 78f541f2 >2014-01-09 15:46:32,000 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host ovirt02. >2014-01-09 15:46:32,033 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host ovirt01. >2014-01-09 15:46:32,319 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-3) Initializing Host: ovirt01 >2014-01-09 15:46:32,319 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) Initializing Host: ovirt02 >2014-01-09 15:46:32,334 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-3) Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 15:46:32,352 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-4) Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 15:46:38,602 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-12) Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 15:46:38,824 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-12) Correlation ID: 1044ecf0, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center CSEnt. Setting status to Non Responsive. >2014-01-09 15:46:38,854 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-12) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 15:46:38,855 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-12) Found spm host bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, host name: ovirt02, according to spmId: 2. >2014-01-09 15:46:38,869 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-12) SpmStatus on vds bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5: SPM >2014-01-09 15:46:38,870 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-12) Using old spm server: ovirt02, no start needed >2014-01-09 15:46:38,877 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-12) Initialize Irs proxy from vds: 192.168.100.22 >2014-01-09 15:46:38,884 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host ovirt02 (Address: 192.168.100.22). >2014-01-09 15:46:39,247 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-6-thread-19) START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false), log id: 5c72c4a2 >2014-01-09 15:46:39,297 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-6-thread-19) -- executeIrsBrokerCommand: Attempting on storage pool 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 15:46:39,306 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (pool-6-thread-19) START, HSMGetAllTasksInfoVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5), log id: 237dc32c >2014-01-09 15:46:39,319 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (pool-6-thread-19) FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 237dc32c >2014-01-09 15:46:39,320 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-6-thread-19) FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 5c72c4a2 >2014-01-09 15:46:39,321 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (pool-6-thread-19) Discovered no tasks on Storage Pool CSEnt >2014-01-09 15:48:09,632 INFO [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-1) Running command: LoginAdminUserCommand internal: false. >2014-01-09 15:48:09,652 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User ryan-a logged in. >2014-01-09 15:48:10,836 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp--127.0.0.1-8702-6) Running command: LoginUserCommand internal: false. >2014-01-09 15:48:10,844 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User ryan-a logged in. >2014-01-09 15:48:57,018 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp--127.0.0.1-8702-5) [7d582990] Lock Acquired to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 15:48:57,195 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-49) [7d582990] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: b6972001-2bd6-4cea-9c15-c046674754d4 Type: VM >2014-01-09 15:48:57,218 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-49) [7d582990] START, MigrateVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=b6972001-2bd6-4cea-9c15-c046674754d4, srcHost=192.168.100.22, dstVdsId=f2d7a8a9-e77e-4e9a-8031-23a0731407c8, dstHost=192.168.100.21:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 10dd9358 >2014-01-09 15:48:57,220 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-49) [7d582990] VdsBroker::migrate::Entered (vm_guid=b6972001-2bd6-4cea-9c15-c046674754d4, srcHost=192.168.100.22, dstHost=192.168.100.21:54321, method=online >2014-01-09 15:48:57,222 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-49) [7d582990] START, MigrateBrokerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=b6972001-2bd6-4cea-9c15-c046674754d4, srcHost=192.168.100.22, dstVdsId=f2d7a8a9-e77e-4e9a-8031-23a0731407c8, dstHost=192.168.100.21:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 71dd67fb >2014-01-09 15:48:57,235 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-49) [7d582990] FINISH, MigrateBrokerVDSCommand, log id: 71dd67fb >2014-01-09 15:48:57,242 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-49) [7d582990] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 10dd9358 >2014-01-09 15:48:57,267 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-49) [7d582990] Correlation ID: 7d582990, Job ID: 475a27e2-362b-46bb-90ec-35ac0a30631a, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: DC-01, Source: ovirt02, Destination: ovirt01, User: ryan-a). >2014-01-09 15:48:57,966 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 15:49:00,998 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-8) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 15:49:04,153 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-11) START, FullListVdsCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02], vmIds=[b6972001-2bd6-4cea-9c15-c046674754d4]), log id: 70b5f0fd >2014-01-09 15:49:04,165 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-14) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 15:49:04,169 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-11) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@51d3d9fe, log id: 70b5f0fd >2014-01-09 15:49:07,224 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 15:49:10,251 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-20) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 15:49:13,278 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-21) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 15:49:16,305 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-27) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 15:49:19,357 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-29) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 15:49:22,400 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-36) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 15:49:25,462 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-38) START, FullListVdsCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01], vmIds=[b6972001-2bd6-4cea-9c15-c046674754d4]), log id: 71e695fb >2014-01-09 15:49:25,475 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-38) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@25b727b9, log id: 71e695fb >2014-01-09 15:49:25,501 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-38) Correlation ID: 7d582990, Job ID: 475a27e2-362b-46bb-90ec-35ac0a30631a, Call Stack: null, Custom Event ID: -1, Message: Migration completed (VM: DC-01, Source: ovirt02, Destination: ovirt01, Duration: 28 sec). >2014-01-09 15:49:25,513 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (DefaultQuartzScheduler_Worker-38) Lock freed to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 15:49:28,491 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-44) START, DestroyVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=b6972001-2bd6-4cea-9c15-c046674754d4, force=false, secondsToWait=0, gracefully=false), log id: 192225b4 >2014-01-09 15:49:29,009 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-44) FINISH, DestroyVDSCommand, log id: 192225b4 >2014-01-09 15:49:29,013 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 status = Down on vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 15:52:11,660 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp--127.0.0.1-8702-2) [5b0525ed] Lock Acquired to object EngineLock [exclusiveLocks= key: 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa value: VM >, sharedLocks= ] >2014-01-09 15:52:11,695 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-49) [5b0525ed] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa Type: VM >2014-01-09 15:52:11,711 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-49) [5b0525ed] START, MigrateVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, srcHost=192.168.100.22, dstVdsId=f2d7a8a9-e77e-4e9a-8031-23a0731407c8, dstHost=192.168.100.21:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 66099090 >2014-01-09 15:52:11,713 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-49) [5b0525ed] VdsBroker::migrate::Entered (vm_guid=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, srcHost=192.168.100.22, dstHost=192.168.100.21:54321, method=online >2014-01-09 15:52:11,715 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-49) [5b0525ed] START, MigrateBrokerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, srcHost=192.168.100.22, dstVdsId=f2d7a8a9-e77e-4e9a-8031-23a0731407c8, dstHost=192.168.100.21:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 34f5205a >2014-01-09 15:52:11,726 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-49) [5b0525ed] FINISH, MigrateBrokerVDSCommand, log id: 34f5205a >2014-01-09 15:52:11,733 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-49) [5b0525ed] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 66099090 >2014-01-09 15:52:11,740 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-49) [5b0525ed] Correlation ID: 5b0525ed, Job ID: fad93c9d-d371-4ca1-957a-b3899895144e, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: VETS, Source: ovirt02, Destination: ovirt01, User: ryan-a). >2014-01-09 15:52:12,084 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) RefreshVmList vm id 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 15:52:15,109 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) RefreshVmList vm id 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 15:52:18,133 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) RefreshVmList vm id 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 15:52:21,178 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) RefreshVmList vm id 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 15:52:24,226 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-89) START, FullListVdsCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01], vmIds=[60ace091-8a4e-4047-9ac0-c0ea95e3c5fa]), log id: 79f74d47 >2014-01-09 15:52:24,237 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-89) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@72b6e11e, log id: 79f74d47 >2014-01-09 15:52:24,240 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-89) Received a memballoon Device without an address when processing VM 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa devices, skipping device: {alias=balloon0, specParams={model=none}, target=393216, device=memballoon, type=balloon} >2014-01-09 15:52:24,257 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-89) Correlation ID: 5b0525ed, Job ID: fad93c9d-d371-4ca1-957a-b3899895144e, Call Stack: null, Custom Event ID: -1, Message: Migration completed (VM: VETS, Source: ovirt02, Destination: ovirt01, Duration: 12 sec). >2014-01-09 15:52:24,263 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (DefaultQuartzScheduler_Worker-89) Lock freed to object EngineLock [exclusiveLocks= key: 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa value: VM >, sharedLocks= ] >2014-01-09 15:52:24,305 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-90) START, DestroyVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, force=false, secondsToWait=0, gracefully=false), log id: 30a1dee6 >2014-01-09 15:52:24,758 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-90) FINISH, DestroyVDSCommand, log id: 30a1dee6 >2014-01-09 15:52:24,761 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-90) RefreshVmList vm id 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa status = Down on vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 15:55:39,359 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (ajp--127.0.0.1-8702-5) [69a8b62] Lock Acquired to object EngineLock [exclusiveLocks= , sharedLocks= key: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d value: POOL >] >2014-01-09 15:55:39,390 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (pool-6-thread-49) [69a8b62] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 15:55:39,395 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-6-thread-49) [69a8b62] START, SetVdsStatusVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, status=PreparingForMaintenance, nonOperationalReason=NONE), log id: 43474d09 >2014-01-09 15:55:39,400 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-6-thread-49) [69a8b62] VDS ovirt02 is spm and moved from up calling ResetIrs. >2014-01-09 15:55:39,401 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (pool-6-thread-49) [69a8b62] START, ResetIrsVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, vdsId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, ignoreStopFailed = false), log id: 1863c38e >2014-01-09 15:55:39,406 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-6-thread-49) [69a8b62] START, SpmStopVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 6be8e25f >2014-01-09 15:55:39,421 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-6-thread-49) [69a8b62] SpmStopVDSCommand::Stopping SPM on vds ovirt02, pool id 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 15:55:42,794 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-6-thread-49) [69a8b62] FINISH, SpmStopVDSCommand, log id: 6be8e25f >2014-01-09 15:55:42,801 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (pool-6-thread-49) [69a8b62] FINISH, ResetIrsVDSCommand, log id: 1863c38e >2014-01-09 15:55:42,802 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-6-thread-49) [69a8b62] FINISH, SetVdsStatusVDSCommand, log id: 43474d09 >2014-01-09 15:55:42,803 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (pool-6-thread-49) [69a8b62] Lock freed to object EngineLock [exclusiveLocks= , sharedLocks= key: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d value: POOL >] >2014-01-09 15:55:42,805 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-65) IrsBroker::Failed::GetStoragePoolInfoVDS due to: SocketException: Socket closed >2014-01-09 15:55:42,815 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-65) IRS failover failed - cant allocate vds server >2014-01-09 15:55:42,838 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-63) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 : ovirt02 >2014-01-09 15:55:42,847 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-6-thread-50) Clearing cache of pool: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d for problematic entities of VDS: ovirt02. >2014-01-09 15:55:42,851 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-63) START, DisconnectStoragePoolVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, vds_spm_id = 2), log id: 2568d679 >2014-01-09 15:55:42,874 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (pool-6-thread-49) [69a8b62] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 15:55:42,880 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-49) [69a8b62] Correlation ID: 69a8b62, Job ID: b19ec2fe-13bf-47ca-bab0-f7fff7b58714, Call Stack: null, Custom Event ID: -1, Message: Host ovirt02 was switched to Maintenance mode by ryan-a. >2014-01-09 15:55:45,762 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-63) FINISH, DisconnectStoragePoolVDSCommand, log id: 2568d679 >2014-01-09 15:55:45,768 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-63) Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 15:55:45,896 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-63) START, DisconnectStorageServerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = ISCSI, connectionList = [{ id: 623aa506-12d1-4437-8e81-f39b2317f3dd, connection: 192.168.101.101, iqn: iqn.2013-11.local.cytechservices.n16000pro:iscsi.hosts-tgt00.raid0, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 4259c6a0 >2014-01-09 15:55:49,074 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-63) FINISH, DisconnectStorageServerVDSCommand, return: {623aa506-12d1-4437-8e81-f39b2317f3dd=0}, log id: 4259c6a0 >2014-01-09 15:55:49,077 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-63) START, DisconnectStorageServerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 210997b7-9098-48d6-8e97-9689085aa4b0, connection: 192.168.101.101:/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: AUTO, nfsRetrans: null, nfsTimeo: null };]), log id: 3a3b606c >2014-01-09 15:55:52,818 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-74) Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 15:55:52,825 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-74) Correlation ID: 76c5deaa, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center CSEnt. Setting status to Non Responsive. >2014-01-09 15:55:52,839 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-74) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 15:55:52,848 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-74) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 15:55:52,864 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-74) SPM selection - vds seems as spm ovirt02 >2014-01-09 15:55:52,866 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-74) START, SpmStopVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 4977cc64 >2014-01-09 15:55:52,885 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand return value > >TaskStatusListReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=654, mMessage=Not SPM: ()]] > >2014-01-09 15:55:52,887 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-74) HostName = ovirt02 >2014-01-09 15:55:52,888 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command HSMGetAllTasksStatusesVDS execution failed. Exception: IRSNonOperationalException: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM: () >2014-01-09 15:55:52,888 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, SpmStopVDSCommand, log id: 4977cc64 >2014-01-09 15:55:52,889 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-74) spm stop succeeded, continuing with spm selection >2014-01-09 15:55:52,892 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-74) starting spm on vds ovirt01, storage pool CSEnt, prevId 2, LVER 12 >2014-01-09 15:55:52,895 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-74) START, SpmStartVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, prevId=2, prevLVER=12, storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=true), log id: 3f1479fc >2014-01-09 15:55:52,910 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-74) spmStart polling started: taskId = 81907813-ce43-41d1-8620-f27f4faed96f >2014-01-09 15:55:54,493 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-63) FINISH, DisconnectStorageServerVDSCommand, return: {210997b7-9098-48d6-8e97-9689085aa4b0=0}, log id: 3a3b606c >2014-01-09 15:55:54,495 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-63) START, DisconnectStorageServerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 4e5d4030-1bd2-4487-b08f-2135eba3f11f, connection: 192.168.101.101:/vms, iqn: null, vfsType: null, mountOptions: null, nfsVersion: V4, nfsRetrans: null, nfsTimeo: null };]), log id: 24fd9018 >2014-01-09 15:55:56,994 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-74) spmStart polling ended: taskId = 81907813-ce43-41d1-8620-f27f4faed96f task status = finished >2014-01-09 15:55:57,004 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-74) spmStart polling ended, spm status: SPM >2014-01-09 15:55:57,007 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-74) START, HSMClearTaskVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, taskId=81907813-ce43-41d1-8620-f27f4faed96f), log id: 4a7c8c56 >2014-01-09 15:55:57,017 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, HSMClearTaskVDSCommand, log id: 4a7c8c56 >2014-01-09 15:55:57,017 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-74) FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.businessentities.SpmStatusResult@341d2f21, log id: 3f1479fc >2014-01-09 15:55:57,020 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-74) Initialize Irs proxy from vds: 192.168.100.21 >2014-01-09 15:55:57,024 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-74) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host ovirt01 (Address: 192.168.100.21). >2014-01-09 15:55:57,033 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-6-thread-49) START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false), log id: 1bcda101 >2014-01-09 15:55:57,055 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-6-thread-49) -- executeIrsBrokerCommand: Attempting on storage pool 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 15:55:57,058 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (pool-6-thread-49) START, HSMGetAllTasksInfoVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8), log id: 144d39e8 >2014-01-09 15:55:57,068 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (pool-6-thread-49) FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 144d39e8 >2014-01-09 15:55:57,068 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-6-thread-49) FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 1bcda101 >2014-01-09 15:55:57,069 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (pool-6-thread-49) Discovered no tasks on Storage Pool CSEnt >2014-01-09 15:55:57,637 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-63) FINISH, DisconnectStorageServerVDSCommand, return: {4e5d4030-1bd2-4487-b08f-2135eba3f11f=0}, log id: 24fd9018 >2014-01-09 16:01:03,310 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (ajp--127.0.0.1-8702-6) [2aa509cf] Lock Acquired to object EngineLock [exclusiveLocks= key: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 value: VDS_FENCE >, sharedLocks= ] >2014-01-09 16:01:03,312 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (pool-6-thread-48) [2aa509cf] Running command: FenceVdsManualyCommand internal: false. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 16:01:03,320 INFO [org.ovirt.engine.core.bll.ClearNonResponsiveVdsVmsCommand] (pool-6-thread-48) Running command: ClearNonResponsiveVdsVmsCommand internal: true. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 16:01:03,325 INFO [org.ovirt.engine.core.vdsbroker.UpdateVdsVMsClearedVDSCommand] (pool-6-thread-48) START, UpdateVdsVMsClearedVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5), log id: 58a79aa3 >2014-01-09 16:01:03,329 INFO [org.ovirt.engine.core.vdsbroker.UpdateVdsVMsClearedVDSCommand] (pool-6-thread-48) FINISH, UpdateVdsVMsClearedVDSCommand, log id: 58a79aa3 >2014-01-09 16:01:03,340 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) Correlation ID: 4a03e74c, Call Stack: null, Custom Event ID: -1, Message: All VMs' status on Non Responsive Host ovirt02 were changed to 'Down' by ryan-a >2014-01-09 16:01:03,353 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) Correlation ID: 2aa509cf, Call Stack: null, Custom Event ID: -1, Message: Manual fence for host ovirt02 was started. >2014-01-09 16:01:03,354 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (pool-6-thread-48) Lock freed to object EngineLock [exclusiveLocks= key: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 value: VDS_FENCE >, sharedLocks= ] >2014-01-09 16:01:06,064 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp--127.0.0.1-8702-7) [8f84811] Lock Acquired to object EngineLock [exclusiveLocks= key: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 value: VDS >, sharedLocks= ] >2014-01-09 16:01:06,078 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-6-thread-48) [8f84811] Running command: ActivateVdsCommand internal: false. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 16:01:06,084 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-6-thread-48) [8f84811] Before acquiring lock in order to prevent monitoring for host ovirt02 from data-center CSEnt >2014-01-09 16:01:06,085 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-6-thread-48) [8f84811] Lock acquired, from now a monitoring of host will be skipped for host ovirt02 from data-center CSEnt >2014-01-09 16:01:06,092 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-6-thread-48) [8f84811] START, SetVdsStatusVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, status=Unassigned, nonOperationalReason=NONE), log id: 6d04dc2a >2014-01-09 16:01:06,097 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-6-thread-48) [8f84811] FINISH, SetVdsStatusVDSCommand, log id: 6d04dc2a >2014-01-09 16:01:06,100 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (pool-6-thread-48) [8f84811] START, ActivateVdsVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5), log id: 79816887 >2014-01-09 16:01:06,465 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (pool-6-thread-48) [8f84811] START, GetHardwareInfoVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02]), log id: 5af8015b >2014-01-09 16:01:06,637 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (pool-6-thread-48) [8f84811] FINISH, GetHardwareInfoVDSCommand, log id: 5af8015b >2014-01-09 16:01:06,652 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (pool-6-thread-48) Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 16:01:06,655 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (pool-6-thread-48) Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 16:01:06,657 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (pool-6-thread-48) FINISH, ActivateVdsVDSCommand, return: Host[ovirt02], log id: 79816887 >2014-01-09 16:01:06,768 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-6-thread-48) Activate finished. Lock released. Monitoring can run now for host ovirt02 from data-center CSEnt >2014-01-09 16:01:06,773 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) Correlation ID: 8f84811, Job ID: b69bc839-1e4e-4066-8faa-7f45378ff995, Call Stack: null, Custom Event ID: -1, Message: Host ovirt02 was activated by ryan-a. >2014-01-09 16:01:06,776 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-6-thread-48) Lock freed to object EngineLock [exclusiveLocks= key: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 value: VDS >, sharedLocks= ] >2014-01-09 16:01:06,953 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-13) START, GetHardwareInfoVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02]), log id: 728035fc >2014-01-09 16:01:06,963 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-13) FINISH, GetHardwareInfoVDSCommand, log id: 728035fc >2014-01-09 16:01:06,974 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-13) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host ovirt02. >2014-01-09 16:01:06,986 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-13) Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 16:01:06,991 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-13) Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 16:01:07,076 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-13) START, ConnectStorageServerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = ISCSI, connectionList = [{ id: 623aa506-12d1-4437-8e81-f39b2317f3dd, connection: 192.168.101.101, iqn: iqn.2013-11.local.cytechservices.n16000pro:iscsi.hosts-tgt00.raid0, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 505a9274 >2014-01-09 16:01:11,093 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-13) FINISH, ConnectStorageServerVDSCommand, return: {623aa506-12d1-4437-8e81-f39b2317f3dd=0}, log id: 505a9274 >2014-01-09 16:01:11,095 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-13) Host ovirt02 storage connection was succeeded >2014-01-09 16:01:11,096 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-13) START, ConnectStorageServerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 210997b7-9098-48d6-8e97-9689085aa4b0, connection: 192.168.101.101:/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: AUTO, nfsRetrans: null, nfsTimeo: null };]), log id: 2b0035d9 >2014-01-09 16:01:11,436 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-13) FINISH, ConnectStorageServerVDSCommand, return: {210997b7-9098-48d6-8e97-9689085aa4b0=0}, log id: 2b0035d9 >2014-01-09 16:01:11,437 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-13) Host ovirt02 storage connection was succeeded >2014-01-09 16:01:11,438 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-13) START, ConnectStorageServerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 4e5d4030-1bd2-4487-b08f-2135eba3f11f, connection: 192.168.101.101:/vms, iqn: null, vfsType: null, mountOptions: null, nfsVersion: V4, nfsRetrans: null, nfsTimeo: null };]), log id: 5a682437 >2014-01-09 16:01:11,509 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-13) FINISH, ConnectStorageServerVDSCommand, return: {4e5d4030-1bd2-4487-b08f-2135eba3f11f=0}, log id: 5a682437 >2014-01-09 16:01:11,509 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-13) Host ovirt02 storage connection was succeeded >2014-01-09 16:01:11,520 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-6-thread-49) START, ConnectStoragePoolVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, vds_spm_id = 2, masterDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, masterVersion = 2), log id: feb1f18 >2014-01-09 16:01:19,485 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-6-thread-49) FINISH, ConnectStoragePoolVDSCommand, log id: feb1f18 >2014-01-09 16:01:19,505 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-13) START, SetMOMPolicyParametersVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5), log id: 3f7d0c5c >2014-01-09 16:01:19,582 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-13) FINISH, SetMOMPolicyParametersVDSCommand, log id: 3f7d0c5c >2014-01-09 16:01:19,586 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-13) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host ovirt02. >2014-01-09 16:01:19,592 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-13) Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 16:01:19,597 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-13) Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 16:02:51,428 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp--127.0.0.1-8702-1) [1feedc7] Lock Acquired to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 16:02:51,477 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-48) [1feedc7] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: 33f88d50-30eb-45c7-9850-733da17ed7a4 Type: VM >2014-01-09 16:02:51,494 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-48) [1feedc7] START, MigrateVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, srcHost=192.168.100.21, dstVdsId=bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, dstHost=192.168.100.22:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 605b7b67 >2014-01-09 16:02:51,497 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [1feedc7] VdsBroker::migrate::Entered (vm_guid=33f88d50-30eb-45c7-9850-733da17ed7a4, srcHost=192.168.100.21, dstHost=192.168.100.22:54321, method=online >2014-01-09 16:02:51,499 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [1feedc7] START, MigrateBrokerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, srcHost=192.168.100.21, dstVdsId=bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, dstHost=192.168.100.22:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 6c114a72 >2014-01-09 16:02:51,509 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-48) [1feedc7] FINISH, MigrateBrokerVDSCommand, log id: 6c114a72 >2014-01-09 16:02:51,515 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-48) [1feedc7] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 605b7b67 >2014-01-09 16:02:51,525 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) [1feedc7] Correlation ID: 1feedc7, Job ID: c18ee1da-ba3c-4e84-b55d-467cafc591d3, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: DC-02, Source: ovirt01, Destination: ovirt02, User: ryan-a). >2014-01-09 16:02:53,112 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:02:56,138 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:02:59,163 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:03:02,183 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-69) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:03:05,203 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:03:08,232 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-78) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:03:11,259 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-63) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:03:14,279 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-86) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:03:17,299 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:03:20,326 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-91) START, FullListVdsCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02], vmIds=[33f88d50-30eb-45c7-9850-733da17ed7a4]), log id: 43354543 >2014-01-09 16:03:20,337 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-91) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@75b086bd, log id: 43354543 >2014-01-09 16:03:20,356 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-91) Correlation ID: 1feedc7, Job ID: c18ee1da-ba3c-4e84-b55d-467cafc591d3, Call Stack: null, Custom Event ID: -1, Message: Migration completed (VM: DC-02, Source: ovirt01, Destination: ovirt02, Duration: 28 sec). >2014-01-09 16:03:20,360 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (DefaultQuartzScheduler_Worker-91) Lock freed to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 16:03:20,829 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-93) START, DestroyVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, force=false, secondsToWait=0, gracefully=false), log id: d36d598 >2014-01-09 16:03:21,389 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-93) FINISH, DestroyVDSCommand, log id: d36d598 >2014-01-09 16:03:21,391 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) RefreshVmList vm id 33f88d50-30eb-45c7-9850-733da17ed7a4 status = Down on vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 16:06:08,271 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp--127.0.0.1-8702-5) [510b350d] Lock Acquired to object EngineLock [exclusiveLocks= key: 872c3dea-6984-48fa-99c3-354fd367c3f9 value: VM >, sharedLocks= ] >2014-01-09 16:06:08,298 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-49) [510b350d] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: 872c3dea-6984-48fa-99c3-354fd367c3f9 Type: VM >2014-01-09 16:06:08,313 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-49) [510b350d] START, MigrateVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, srcHost=192.168.100.21, dstVdsId=bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, dstHost=192.168.100.22:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 53fc5960 >2014-01-09 16:06:08,316 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-49) [510b350d] VdsBroker::migrate::Entered (vm_guid=872c3dea-6984-48fa-99c3-354fd367c3f9, srcHost=192.168.100.21, dstHost=192.168.100.22:54321, method=online >2014-01-09 16:06:08,318 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-49) [510b350d] START, MigrateBrokerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, srcHost=192.168.100.21, dstVdsId=bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, dstHost=192.168.100.22:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 2be545ee >2014-01-09 16:06:08,328 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-49) [510b350d] FINISH, MigrateBrokerVDSCommand, log id: 2be545ee >2014-01-09 16:06:08,334 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-49) [510b350d] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 53fc5960 >2014-01-09 16:06:08,339 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-49) [510b350d] Correlation ID: 510b350d, Job ID: 503bc1ab-4a92-4f8a-9741-01984df7fe77, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: External_Email, Source: ovirt01, Destination: ovirt02, User: ryan-a). >2014-01-09 16:06:09,673 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:06:12,701 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-29) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:06:13,693 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-32) START, FullListVdsCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01], vmIds=[872c3dea-6984-48fa-99c3-354fd367c3f9]), log id: 405376d0 >2014-01-09 16:06:13,705 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-32) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@51d411d3, log id: 405376d0 >2014-01-09 16:06:15,724 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-34) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:06:18,744 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-36) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:06:21,764 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-43) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:06:24,797 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-49) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:06:27,826 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-53) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:06:30,846 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-61) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:06:33,865 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-64) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:06:36,884 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-69) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:06:39,922 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:06:42,951 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:06:45,971 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:06:49,001 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:06:52,021 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-87) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:06:55,057 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-91) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:06:58,092 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-95) START, FullListVdsCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02], vmIds=[872c3dea-6984-48fa-99c3-354fd367c3f9]), log id: 20df3a39 >2014-01-09 16:06:58,102 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-95) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@454fa548, log id: 20df3a39 >2014-01-09 16:06:58,116 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-95) Correlation ID: 510b350d, Job ID: 503bc1ab-4a92-4f8a-9741-01984df7fe77, Call Stack: null, Custom Event ID: -1, Message: Migration completed (VM: External_Email, Source: ovirt01, Destination: ovirt02, Duration: 49 sec). >2014-01-09 16:06:58,119 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (DefaultQuartzScheduler_Worker-95) Lock freed to object EngineLock [exclusiveLocks= key: 872c3dea-6984-48fa-99c3-354fd367c3f9 value: VM >, sharedLocks= ] >2014-01-09 16:06:59,049 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-100) START, DestroyVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, force=false, secondsToWait=0, gracefully=false), log id: 69dcc39e >2014-01-09 16:06:59,540 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-100) FINISH, DestroyVDSCommand, log id: 69dcc39e >2014-01-09 16:06:59,550 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) RefreshVmList vm id 872c3dea-6984-48fa-99c3-354fd367c3f9 status = Down on vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 16:07:29,518 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp--127.0.0.1-8702-4) [412c340c] Lock Acquired to object EngineLock [exclusiveLocks= key: 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa value: VM >, sharedLocks= ] >2014-01-09 16:07:29,570 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-49) [412c340c] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa Type: VM >2014-01-09 16:07:29,582 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-49) [412c340c] START, MigrateVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, srcHost=192.168.100.21, dstVdsId=bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, dstHost=192.168.100.22:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 109ad56d >2014-01-09 16:07:29,583 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-49) [412c340c] VdsBroker::migrate::Entered (vm_guid=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, srcHost=192.168.100.21, dstHost=192.168.100.22:54321, method=online >2014-01-09 16:07:29,585 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-49) [412c340c] START, MigrateBrokerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, srcHost=192.168.100.21, dstVdsId=bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, dstHost=192.168.100.22:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 1b16c51d >2014-01-09 16:07:29,602 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-49) [412c340c] FINISH, MigrateBrokerVDSCommand, log id: 1b16c51d >2014-01-09 16:07:29,607 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-49) [412c340c] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 109ad56d >2014-01-09 16:07:29,614 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-49) [412c340c] Correlation ID: 412c340c, Job ID: 24623764-6cdc-475b-b453-7e6062f266ab, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: VETS, Source: ovirt01, Destination: ovirt02, User: ryan-a). >2014-01-09 16:07:31,359 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-49) RefreshVmList vm id 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:07:34,380 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-48) RefreshVmList vm id 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:07:37,400 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-53) RefreshVmList vm id 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:07:40,478 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-57) RefreshVmList vm id 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:07:41,841 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-62) START, DestroyVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, force=false, secondsToWait=0, gracefully=false), log id: deb47f5 >2014-01-09 16:07:42,335 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-62) FINISH, DestroyVDSCommand, log id: deb47f5 >2014-01-09 16:07:42,338 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) Handing over VM VETS 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa to Host bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5. Setting VM to status MigratingTo >2014-01-09 16:07:42,339 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) vm VETS running in db and not running in vds - add to rerun treatment. vds ovirt01 >2014-01-09 16:07:43,522 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-59) VM VETS 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa moved from MigratingTo --> Up >2014-01-09 16:07:43,523 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-59) START, FullListVdsCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02], vmIds=[60ace091-8a4e-4047-9ac0-c0ea95e3c5fa]), log id: 5c70558b >2014-01-09 16:07:43,532 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-59) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@377cc6ec, log id: 5c70558b >2014-01-09 16:07:43,534 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-59) Received a memballoon Device without an address when processing VM 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa devices, skipping device: {alias=balloon0, specParams={model=none}, target=393216, device=memballoon, type=balloon} >2014-01-09 16:07:43,546 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) Correlation ID: 412c340c, Job ID: 24623764-6cdc-475b-b453-7e6062f266ab, Call Stack: null, Custom Event ID: -1, Message: Migration completed (VM: VETS, Source: ovirt01, Destination: ovirt02, Duration: 13 sec). >2014-01-09 16:07:43,549 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (DefaultQuartzScheduler_Worker-59) Lock freed to object EngineLock [exclusiveLocks= key: 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa value: VM >, sharedLocks= ] >2014-01-09 16:08:01,100 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp--127.0.0.1-8702-8) [6c1cd786] Lock Acquired to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 16:08:01,129 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-49) [6c1cd786] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: b6972001-2bd6-4cea-9c15-c046674754d4 Type: VM >2014-01-09 16:08:01,144 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-49) [6c1cd786] START, MigrateVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=b6972001-2bd6-4cea-9c15-c046674754d4, srcHost=192.168.100.21, dstVdsId=bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, dstHost=192.168.100.22:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 3fb66598 >2014-01-09 16:08:01,145 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-49) [6c1cd786] VdsBroker::migrate::Entered (vm_guid=b6972001-2bd6-4cea-9c15-c046674754d4, srcHost=192.168.100.21, dstHost=192.168.100.22:54321, method=online >2014-01-09 16:08:01,147 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-49) [6c1cd786] START, MigrateBrokerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=b6972001-2bd6-4cea-9c15-c046674754d4, srcHost=192.168.100.21, dstVdsId=bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, dstHost=192.168.100.22:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 62608ef2 >2014-01-09 16:08:01,181 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-49) [6c1cd786] FINISH, MigrateBrokerVDSCommand, log id: 62608ef2 >2014-01-09 16:08:01,186 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-49) [6c1cd786] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 3fb66598 >2014-01-09 16:08:01,191 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-49) [6c1cd786] Correlation ID: 6c1cd786, Job ID: 985ce277-b872-4c5d-8516-210b60b21a00, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: DC-01, Source: ovirt01, Destination: ovirt02, User: ryan-a). >2014-01-09 16:08:01,672 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:08:04,691 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-92) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:08:07,711 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:08:10,756 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:08:13,807 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:08:16,829 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:08:19,849 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-8) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:08:22,869 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:08:25,909 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-17) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:08:28,949 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-21) START, FullListVdsCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02], vmIds=[b6972001-2bd6-4cea-9c15-c046674754d4]), log id: f19427b >2014-01-09 16:08:28,958 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-21) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@34a5e149, log id: f19427b >2014-01-09 16:08:28,971 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) Correlation ID: 6c1cd786, Job ID: 985ce277-b872-4c5d-8516-210b60b21a00, Call Stack: null, Custom Event ID: -1, Message: Migration completed (VM: DC-01, Source: ovirt01, Destination: ovirt02, Duration: 27 sec). >2014-01-09 16:08:28,974 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (DefaultQuartzScheduler_Worker-21) Lock freed to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 16:08:32,682 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-19) START, DestroyVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=b6972001-2bd6-4cea-9c15-c046674754d4, force=false, secondsToWait=0, gracefully=false), log id: 1321a5fb >2014-01-09 16:08:33,168 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-19) FINISH, DestroyVDSCommand, log id: 1321a5fb >2014-01-09 16:08:33,172 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-19) Handing over VM DC-01 b6972001-2bd6-4cea-9c15-c046674754d4 to Host bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5. Setting VM to status MigratingTo >2014-01-09 16:08:33,173 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-19) vm DC-01 running in db and not running in vds - add to rerun treatment. vds ovirt01 >2014-01-09 16:08:35,008 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-32) VM DC-01 b6972001-2bd6-4cea-9c15-c046674754d4 moved from MigratingTo --> Up >2014-01-09 16:08:35,009 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-32) START, FullListVdsCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02], vmIds=[b6972001-2bd6-4cea-9c15-c046674754d4]), log id: 46645a66 >2014-01-09 16:08:35,018 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-32) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@7b7a9183, log id: 46645a66 >2014-01-09 16:09:42,105 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (ajp--127.0.0.1-8702-10) [1b87c3fa] Lock Acquired to object EngineLock [exclusiveLocks= , sharedLocks= key: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d value: POOL >] >2014-01-09 16:09:42,127 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (pool-6-thread-49) [1b87c3fa] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 16:09:42,130 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-6-thread-49) [1b87c3fa] START, SetVdsStatusVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, status=PreparingForMaintenance, nonOperationalReason=NONE), log id: 3afa848e >2014-01-09 16:09:42,134 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-6-thread-49) [1b87c3fa] VDS ovirt01 is spm and moved from up calling ResetIrs. >2014-01-09 16:09:42,135 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (pool-6-thread-49) [1b87c3fa] START, ResetIrsVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, vdsId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, ignoreStopFailed = false), log id: 4b8490e7 >2014-01-09 16:09:42,138 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-6-thread-49) [1b87c3fa] START, SpmStopVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 6807c31 >2014-01-09 16:09:42,148 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-6-thread-49) [1b87c3fa] SpmStopVDSCommand::Stopping SPM on vds ovirt01, pool id 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 16:09:45,461 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (pool-6-thread-49) [1b87c3fa] FINISH, SpmStopVDSCommand, log id: 6807c31 >2014-01-09 16:09:45,466 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) IrsBroker::Failed::GetStoragePoolInfoVDS due to: SocketException: Socket closed >2014-01-09 16:09:45,468 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (pool-6-thread-49) [1b87c3fa] FINISH, ResetIrsVDSCommand, log id: 4b8490e7 >2014-01-09 16:09:45,468 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) IRS failover failed - cant allocate vds server >2014-01-09 16:09:45,468 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-6-thread-49) [1b87c3fa] FINISH, SetVdsStatusVDSCommand, log id: 3afa848e >2014-01-09 16:09:45,470 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (pool-6-thread-49) [1b87c3fa] Lock freed to object EngineLock [exclusiveLocks= , sharedLocks= key: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d value: POOL >] >2014-01-09 16:09:45,486 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (pool-6-thread-49) [1b87c3fa] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 16:09:45,490 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-49) [1b87c3fa] Correlation ID: 1b87c3fa, Job ID: ade2f5cb-ae33-4cf3-82ce-6457ff0ee162, Call Stack: null, Custom Event ID: -1, Message: Host ovirt01 was switched to Maintenance mode by ryan-a. >2014-01-09 16:09:45,533 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-13) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01 >2014-01-09 16:09:45,540 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (pool-6-thread-48) Clearing cache of pool: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d for problematic entities of VDS: ovirt01. >2014-01-09 16:09:45,543 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-13) START, DisconnectStoragePoolVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, vds_spm_id = 1), log id: 7c859c0d >2014-01-09 16:09:50,343 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-13) FINISH, DisconnectStoragePoolVDSCommand, log id: 7c859c0d >2014-01-09 16:09:50,345 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-13) Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 16:09:50,427 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-13) START, DisconnectStorageServerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = ISCSI, connectionList = [{ id: 623aa506-12d1-4437-8e81-f39b2317f3dd, connection: 192.168.101.101, iqn: iqn.2013-11.local.cytechservices.n16000pro:iscsi.hosts-tgt00.raid0, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 65c2a503 >2014-01-09 16:09:54,615 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-13) FINISH, DisconnectStorageServerVDSCommand, return: {623aa506-12d1-4437-8e81-f39b2317f3dd=0}, log id: 65c2a503 >2014-01-09 16:09:54,617 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-13) START, DisconnectStorageServerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 210997b7-9098-48d6-8e97-9689085aa4b0, connection: 192.168.101.101:/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: AUTO, nfsRetrans: null, nfsTimeo: null };]), log id: 6de9a5a9 >2014-01-09 16:09:55,471 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-35) Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 16:09:55,477 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-35) Correlation ID: 3cc38cf3, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center CSEnt. Setting status to Non Responsive. >2014-01-09 16:09:55,490 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-35) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 16:09:55,495 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-35) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 16:09:55,498 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-35) SPM selection - vds seems as spm ovirt01 >2014-01-09 16:09:55,500 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-35) START, SpmStopVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 5de42c3d >2014-01-09 16:09:55,511 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand return value > >TaskStatusListReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=654, mMessage=Not SPM: ()]] > >2014-01-09 16:09:55,512 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-35) HostName = ovirt01 >2014-01-09 16:09:55,513 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command HSMGetAllTasksStatusesVDS execution failed. Exception: IRSNonOperationalException: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM: () >2014-01-09 16:09:55,514 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-35) FINISH, SpmStopVDSCommand, log id: 5de42c3d >2014-01-09 16:09:55,515 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-35) spm stop succeeded, continuing with spm selection >2014-01-09 16:09:55,517 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-35) starting spm on vds ovirt02, storage pool CSEnt, prevId 1, LVER 13 >2014-01-09 16:09:55,518 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-35) START, SpmStartVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, prevId=1, prevLVER=13, storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=true), log id: 32857687 >2014-01-09 16:09:55,532 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-35) spmStart polling started: taskId = 455fa96b-dc29-4898-87b7-3e9fc7c11554 >2014-01-09 16:09:58,565 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-35) spmStart polling ended: taskId = 455fa96b-dc29-4898-87b7-3e9fc7c11554 task status = finished >2014-01-09 16:09:58,573 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-35) spmStart polling ended, spm status: SPM >2014-01-09 16:09:58,574 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-35) START, HSMClearTaskVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, taskId=455fa96b-dc29-4898-87b7-3e9fc7c11554), log id: 6dce1f3a >2014-01-09 16:09:58,583 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-35) FINISH, HSMClearTaskVDSCommand, log id: 6dce1f3a >2014-01-09 16:09:58,583 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-35) FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.businessentities.SpmStatusResult@3798785f, log id: 32857687 >2014-01-09 16:09:58,585 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-35) Initialize Irs proxy from vds: 192.168.100.22 >2014-01-09 16:09:58,589 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-35) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host ovirt02 (Address: 192.168.100.22). >2014-01-09 16:09:58,599 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-6-thread-49) START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false), log id: 78c3eab0 >2014-01-09 16:09:58,621 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-6-thread-49) -- executeIrsBrokerCommand: Attempting on storage pool 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 16:09:58,623 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (pool-6-thread-49) START, HSMGetAllTasksInfoVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5), log id: 1d14d9c1 >2014-01-09 16:09:58,632 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (pool-6-thread-49) FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 1d14d9c1 >2014-01-09 16:09:58,632 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-6-thread-49) FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 78c3eab0 >2014-01-09 16:09:58,633 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (pool-6-thread-49) Discovered no tasks on Storage Pool CSEnt >2014-01-09 16:09:58,761 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-13) FINISH, DisconnectStorageServerVDSCommand, return: {210997b7-9098-48d6-8e97-9689085aa4b0=0}, log id: 6de9a5a9 >2014-01-09 16:09:58,763 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-13) START, DisconnectStorageServerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 4e5d4030-1bd2-4487-b08f-2135eba3f11f, connection: 192.168.101.101:/vms, iqn: null, vfsType: null, mountOptions: null, nfsVersion: V4, nfsRetrans: null, nfsTimeo: null };]), log id: 2daeea5d >2014-01-09 16:10:02,900 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-13) FINISH, DisconnectStorageServerVDSCommand, return: {4e5d4030-1bd2-4487-b08f-2135eba3f11f=0}, log id: 2daeea5d >2014-01-09 16:10:48,091 INFO [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-2) Running command: LoginAdminUserCommand internal: false. >2014-01-09 16:10:48,095 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2014-01-09 16:12:32,909 INFO [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-7) Running command: LoginAdminUserCommand internal: false. >2014-01-09 16:12:32,924 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2014-01-09 16:12:32,961 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp--127.0.0.1-8702-4) Running command: LoginUserCommand internal: false. >2014-01-09 16:12:32,966 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2014-01-09 16:16:14,842 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (ajp--127.0.0.1-8702-6) [3182df47] Lock Acquired to object EngineLock [exclusiveLocks= key: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 value: VDS_FENCE >, sharedLocks= ] >2014-01-09 16:16:14,844 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (pool-6-thread-50) [3182df47] Running command: FenceVdsManualyCommand internal: false. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 16:16:14,849 INFO [org.ovirt.engine.core.bll.ClearNonResponsiveVdsVmsCommand] (pool-6-thread-50) Running command: ClearNonResponsiveVdsVmsCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 16:16:14,851 INFO [org.ovirt.engine.core.vdsbroker.UpdateVdsVMsClearedVDSCommand] (pool-6-thread-50) START, UpdateVdsVMsClearedVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8), log id: 1aead30a >2014-01-09 16:16:14,853 INFO [org.ovirt.engine.core.vdsbroker.UpdateVdsVMsClearedVDSCommand] (pool-6-thread-50) FINISH, UpdateVdsVMsClearedVDSCommand, log id: 1aead30a >2014-01-09 16:16:14,857 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-50) Correlation ID: f6460ad, Call Stack: null, Custom Event ID: -1, Message: All VMs' status on Non Responsive Host ovirt01 were changed to 'Down' by ryan-a >2014-01-09 16:16:14,862 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-50) Correlation ID: 3182df47, Call Stack: null, Custom Event ID: -1, Message: Manual fence for host ovirt01 was started. >2014-01-09 16:16:14,863 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (pool-6-thread-50) Lock freed to object EngineLock [exclusiveLocks= key: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 value: VDS_FENCE >, sharedLocks= ] >2014-01-09 16:16:22,425 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp--127.0.0.1-8702-10) [1a60fc1e] Lock Acquired to object EngineLock [exclusiveLocks= key: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 value: VDS >, sharedLocks= ] >2014-01-09 16:16:22,442 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-6-thread-50) [1a60fc1e] Running command: ActivateVdsCommand internal: false. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 16:16:22,443 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-6-thread-50) [1a60fc1e] Before acquiring lock in order to prevent monitoring for host ovirt01 from data-center CSEnt >2014-01-09 16:16:22,444 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-6-thread-50) [1a60fc1e] Lock acquired, from now a monitoring of host will be skipped for host ovirt01 from data-center CSEnt >2014-01-09 16:16:22,451 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-6-thread-50) [1a60fc1e] START, SetVdsStatusVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, status=Unassigned, nonOperationalReason=NONE), log id: 28ba9fff >2014-01-09 16:16:22,454 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-6-thread-50) [1a60fc1e] FINISH, SetVdsStatusVDSCommand, log id: 28ba9fff >2014-01-09 16:16:22,457 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (pool-6-thread-50) [1a60fc1e] START, ActivateVdsVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8), log id: 590fcced >2014-01-09 16:16:23,362 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (pool-6-thread-50) [1a60fc1e] START, GetHardwareInfoVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01]), log id: 12b22fc6 >2014-01-09 16:16:23,541 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (pool-6-thread-50) [1a60fc1e] FINISH, GetHardwareInfoVDSCommand, log id: 12b22fc6 >2014-01-09 16:16:23,552 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (pool-6-thread-50) Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 16:16:23,555 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (pool-6-thread-50) Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 16:16:23,556 INFO [org.ovirt.engine.core.vdsbroker.ActivateVdsVDSCommand] (pool-6-thread-50) FINISH, ActivateVdsVDSCommand, return: Host[ovirt01], log id: 590fcced >2014-01-09 16:16:23,660 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-6-thread-50) Activate finished. Lock released. Monitoring can run now for host ovirt01 from data-center CSEnt >2014-01-09 16:16:23,663 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-50) Correlation ID: 1a60fc1e, Job ID: 3b1c7f14-de65-441c-998c-602037bf47fa, Call Stack: null, Custom Event ID: -1, Message: Host ovirt01 was activated by ryan-a. >2014-01-09 16:16:23,667 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (pool-6-thread-50) Lock freed to object EngineLock [exclusiveLocks= key: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 value: VDS >, sharedLocks= ] >2014-01-09 16:16:24,278 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-70) START, GetHardwareInfoVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01]), log id: 2dcda979 >2014-01-09 16:16:24,289 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-70) FINISH, GetHardwareInfoVDSCommand, log id: 2dcda979 >2014-01-09 16:16:24,300 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-70) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host ovirt01. >2014-01-09 16:16:24,309 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-70) Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 16:16:24,311 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-70) Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 16:16:24,402 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-70) START, ConnectStorageServerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = ISCSI, connectionList = [{ id: 623aa506-12d1-4437-8e81-f39b2317f3dd, connection: 192.168.101.101, iqn: iqn.2013-11.local.cytechservices.n16000pro:iscsi.hosts-tgt00.raid0, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 397c2ee >2014-01-09 16:16:27,439 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-70) FINISH, ConnectStorageServerVDSCommand, return: {623aa506-12d1-4437-8e81-f39b2317f3dd=0}, log id: 397c2ee >2014-01-09 16:16:27,440 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-70) Host ovirt01 storage connection was succeeded >2014-01-09 16:16:27,443 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-70) START, ConnectStorageServerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 210997b7-9098-48d6-8e97-9689085aa4b0, connection: 192.168.101.101:/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: AUTO, nfsRetrans: null, nfsTimeo: null };]), log id: 29c2a221 >2014-01-09 16:16:27,787 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-70) FINISH, ConnectStorageServerVDSCommand, return: {210997b7-9098-48d6-8e97-9689085aa4b0=0}, log id: 29c2a221 >2014-01-09 16:16:27,788 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-70) Host ovirt01 storage connection was succeeded >2014-01-09 16:16:27,789 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-70) START, ConnectStorageServerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 4e5d4030-1bd2-4487-b08f-2135eba3f11f, connection: 192.168.101.101:/vms, iqn: null, vfsType: null, mountOptions: null, nfsVersion: V4, nfsRetrans: null, nfsTimeo: null };]), log id: ae8425 >2014-01-09 16:16:27,861 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-70) FINISH, ConnectStorageServerVDSCommand, return: {4e5d4030-1bd2-4487-b08f-2135eba3f11f=0}, log id: ae8425 >2014-01-09 16:16:27,862 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-70) Host ovirt01 storage connection was succeeded >2014-01-09 16:16:27,867 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-6-thread-49) START, ConnectStoragePoolVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, vds_spm_id = 1, masterDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, masterVersion = 2), log id: 34db8e62 >2014-01-09 16:16:36,171 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-6-thread-49) FINISH, ConnectStoragePoolVDSCommand, log id: 34db8e62 >2014-01-09 16:16:36,188 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-70) START, SetMOMPolicyParametersVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8), log id: 5dacd4d4 >2014-01-09 16:16:36,265 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-70) FINISH, SetMOMPolicyParametersVDSCommand, log id: 5dacd4d4 >2014-01-09 16:16:36,269 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-70) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host ovirt01. >2014-01-09 16:16:36,275 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-70) Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 16:16:36,278 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-70) Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 16:17:28,603 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (ajp--127.0.0.1-8702-9) [1a6a7f03] Lock Acquired to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 16:17:28,755 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (pool-6-thread-49) Running command: CreateAllSnapshotsFromVmCommand internal: false. Entities affected : ID: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 Type: VM >2014-01-09 16:17:28,904 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (pool-6-thread-49) Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-01-09 16:17:28,921 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (pool-6-thread-49) START, CreateSnapshotVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, storageDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, imageGroupId = de474e2f-5669-47ac-9103-692fd87aad30, imageSizeInBytes = 32212254720, volumeFormat = COW, newImageId = 4fc85985-130a-4ae4-b98d-c10fa86db741, newImageDescription = , imageId = 40742666-bb4f-4741-9d1e-08dddb2dba27, sourceImageGroupId = de474e2f-5669-47ac-9103-692fd87aad30), log id: a9ff959 >2014-01-09 16:17:28,922 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (pool-6-thread-49) -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-01-09 16:17:28,923 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (pool-6-thread-49) -- createVolume parameters: > sdUUID=cbdefcb2-08ca-400f-84e0-1a6c7d28d763 > spUUID=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d > imgGUID=de474e2f-5669-47ac-9103-692fd87aad30 > size=32,212,254,720 bytes > volFormat=COW > volType=Sparse > volUUID=4fc85985-130a-4ae4-b98d-c10fa86db741 > descr= > srcImgGUID=de474e2f-5669-47ac-9103-692fd87aad30 > srcVolUUID=40742666-bb4f-4741-9d1e-08dddb2dba27 > >2014-01-09 16:17:29,336 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (pool-6-thread-49) FINISH, CreateSnapshotVDSCommand, return: 4fc85985-130a-4ae4-b98d-c10fa86db741, log id: a9ff959 >2014-01-09 16:17:29,383 INFO [org.ovirt.engine.core.bll.CommandAsyncTask] (pool-6-thread-49) CommandAsyncTask::Adding CommandMultiAsyncTasks object for command a8795666-ef0b-4314-834c-a43ec0c0a133 >2014-01-09 16:17:29,384 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (pool-6-thread-49) CommandMultiAsyncTasks::AttachTask: Attaching task db3e4ab4-6599-413a-a30d-5209a1eb3e78 to command a8795666-ef0b-4314-834c-a43ec0c0a133. >2014-01-09 16:17:29,398 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (pool-6-thread-49) Adding task db3e4ab4-6599-413a-a30d-5209a1eb3e78 (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-01-09 16:17:29,429 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (pool-6-thread-49) Running command: CreateSnapshotCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-01-09 16:17:29,441 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (pool-6-thread-49) START, CreateSnapshotVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, storageDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, imageGroupId = d8f95faa-2402-4906-b6fe-d560edf56a4a, imageSizeInBytes = 53687091200, volumeFormat = COW, newImageId = 639c65af-616f-4f92-a711-e617019642bb, newImageDescription = , imageId = bd6325e5-934e-446d-a53e-1d5113188611, sourceImageGroupId = d8f95faa-2402-4906-b6fe-d560edf56a4a), log id: 3d235bff >2014-01-09 16:17:29,443 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (pool-6-thread-49) -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-01-09 16:17:29,444 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (pool-6-thread-49) -- createVolume parameters: > sdUUID=cbdefcb2-08ca-400f-84e0-1a6c7d28d763 > spUUID=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d > imgGUID=d8f95faa-2402-4906-b6fe-d560edf56a4a > size=53,687,091,200 bytes > volFormat=COW > volType=Sparse > volUUID=639c65af-616f-4f92-a711-e617019642bb > descr= > srcImgGUID=d8f95faa-2402-4906-b6fe-d560edf56a4a > srcVolUUID=bd6325e5-934e-446d-a53e-1d5113188611 > >2014-01-09 16:17:30,046 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (pool-6-thread-49) FINISH, CreateSnapshotVDSCommand, return: 639c65af-616f-4f92-a711-e617019642bb, log id: 3d235bff >2014-01-09 16:17:30,051 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (pool-6-thread-49) CommandMultiAsyncTasks::AttachTask: Attaching task 7ebba3de-5956-4fb1-88c6-06490cf94b1e to command a8795666-ef0b-4314-834c-a43ec0c0a133. >2014-01-09 16:17:30,056 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (pool-6-thread-49) Adding task 7ebba3de-5956-4fb1-88c6-06490cf94b1e (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-01-09 16:17:30,065 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (pool-6-thread-49) Lock freed to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 16:17:30,080 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-49) Correlation ID: 1a6a7f03, Job ID: 71396624-e915-4bd2-9b72-adfa10911d51, Call Stack: null, Custom Event ID: -1, Message: Snapshot 'test' creation for VM 'syslog00' was initiated by ryan-a. >2014-01-09 16:17:30,081 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (pool-6-thread-49) BaseAsyncTask::StartPollingTask: Starting to poll task db3e4ab4-6599-413a-a30d-5209a1eb3e78. >2014-01-09 16:17:30,082 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (pool-6-thread-49) BaseAsyncTask::StartPollingTask: Starting to poll task 7ebba3de-5956-4fb1-88c6-06490cf94b1e. >2014-01-09 16:17:37,273 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (DefaultQuartzScheduler_Worker-76) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-01-09 16:17:37,284 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-76) SPMAsyncTask::PollTask: Polling task db3e4ab4-6599-413a-a30d-5209a1eb3e78 (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-01-09 16:17:37,293 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-76) BaseAsyncTask::OnTaskEndSuccess: Task db3e4ab4-6599-413a-a30d-5209a1eb3e78 (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-01-09 16:17:37,295 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-76) Task with DB Task ID da25d962-c75b-427c-a970-e92a8958e05a and VDSM Task ID 7ebba3de-5956-4fb1-88c6-06490cf94b1e is in state Polling. End action for command a8795666-ef0b-4314-834c-a43ec0c0a133 will proceed when all the entitys tasks are completed. >2014-01-09 16:17:37,296 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-76) SPMAsyncTask::PollTask: Polling task 7ebba3de-5956-4fb1-88c6-06490cf94b1e (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-01-09 16:17:37,300 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-76) BaseAsyncTask::OnTaskEndSuccess: Task 7ebba3de-5956-4fb1-88c6-06490cf94b1e (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-01-09 16:17:37,301 INFO [org.ovirt.engine.core.bll.CommandAsyncTask] (DefaultQuartzScheduler_Worker-76) CommandAsyncTask::EndActionIfNecessary: All tasks of command a8795666-ef0b-4314-834c-a43ec0c0a133 has ended -> executing EndAction >2014-01-09 16:17:37,302 INFO [org.ovirt.engine.core.bll.CommandAsyncTask] (DefaultQuartzScheduler_Worker-76) CommandAsyncTask::EndAction: Ending action for 2 tasks (command ID: a8795666-ef0b-4314-834c-a43ec0c0a133): calling EndAction . >2014-01-09 16:17:37,304 INFO [org.ovirt.engine.core.bll.CommandAsyncTask] (pool-6-thread-50) CommandAsyncTask::EndCommandAction [within thread] context: Attempting to EndAction CreateAllSnapshotsFromVm, executionIndex: 0 >2014-01-09 16:17:37,318 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (pool-6-thread-50) Ending command successfully: org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand >2014-01-09 16:17:37,332 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (pool-6-thread-50) [20a8557d] Ending command successfully: org.ovirt.engine.core.bll.CreateSnapshotCommand >2014-01-09 16:17:37,344 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (pool-6-thread-50) [20a8557d] START, GetImageInfoVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, storageDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, imageGroupId = de474e2f-5669-47ac-9103-692fd87aad30, imageId = 4fc85985-130a-4ae4-b98d-c10fa86db741), log id: 6e44717d >2014-01-09 16:17:37,824 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (pool-6-thread-50) [20a8557d] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@520f9b80, log id: 6e44717d >2014-01-09 16:17:37,835 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (pool-6-thread-50) [edbaa81] Ending command successfully: org.ovirt.engine.core.bll.CreateSnapshotCommand >2014-01-09 16:17:37,837 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (pool-6-thread-50) [edbaa81] START, GetImageInfoVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, storageDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, imageGroupId = d8f95faa-2402-4906-b6fe-d560edf56a4a, imageId = 639c65af-616f-4f92-a711-e617019642bb), log id: 18b34750 >2014-01-09 16:17:38,073 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (pool-6-thread-50) [edbaa81] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@10591e73, log id: 18b34750 >2014-01-09 16:17:38,079 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (pool-6-thread-50) Trying to release exclusive lock which does not exist, lock key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7VM >2014-01-09 16:17:38,080 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (pool-6-thread-50) Lock freed to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 16:17:38,085 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-50) Correlation ID: 1a6a7f03, Job ID: 71396624-e915-4bd2-9b72-adfa10911d51, Call Stack: null, Custom Event ID: -1, Message: Snapshot 'test' creation for VM 'syslog00' has been completed. >2014-01-09 16:17:38,086 INFO [org.ovirt.engine.core.bll.CommandAsyncTask] (pool-6-thread-50) CommandAsyncTask::HandleEndActionResult [within thread]: EndAction for action type CreateAllSnapshotsFromVm completed, handling the result. >2014-01-09 16:17:38,087 INFO [org.ovirt.engine.core.bll.CommandAsyncTask] (pool-6-thread-50) CommandAsyncTask::HandleEndActionResult [within thread]: EndAction for action type CreateAllSnapshotsFromVm succeeded, clearing tasks. >2014-01-09 16:17:38,090 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (pool-6-thread-50) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 7ebba3de-5956-4fb1-88c6-06490cf94b1e >2014-01-09 16:17:38,091 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (pool-6-thread-50) START, SPMClearTaskVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, taskId = 7ebba3de-5956-4fb1-88c6-06490cf94b1e), log id: 1c99b457 >2014-01-09 16:17:38,093 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (pool-6-thread-50) START, HSMClearTaskVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, taskId=7ebba3de-5956-4fb1-88c6-06490cf94b1e), log id: 2f0eca6c >2014-01-09 16:17:38,104 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (pool-6-thread-50) FINISH, HSMClearTaskVDSCommand, log id: 2f0eca6c >2014-01-09 16:17:38,105 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (pool-6-thread-50) FINISH, SPMClearTaskVDSCommand, log id: 1c99b457 >2014-01-09 16:17:38,110 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (pool-6-thread-50) BaseAsyncTask::RemoveTaskFromDB: Removed task 7ebba3de-5956-4fb1-88c6-06490cf94b1e from DataBase >2014-01-09 16:17:38,111 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (pool-6-thread-50) SPMAsyncTask::ClearAsyncTask: Attempting to clear task db3e4ab4-6599-413a-a30d-5209a1eb3e78 >2014-01-09 16:17:38,112 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (pool-6-thread-50) START, SPMClearTaskVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, taskId = db3e4ab4-6599-413a-a30d-5209a1eb3e78), log id: 6fc89f34 >2014-01-09 16:17:38,113 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (pool-6-thread-50) START, HSMClearTaskVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, taskId=db3e4ab4-6599-413a-a30d-5209a1eb3e78), log id: 5c4c4841 >2014-01-09 16:17:38,123 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (pool-6-thread-50) FINISH, HSMClearTaskVDSCommand, log id: 5c4c4841 >2014-01-09 16:17:38,123 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (pool-6-thread-50) FINISH, SPMClearTaskVDSCommand, log id: 6fc89f34 >2014-01-09 16:17:38,125 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (pool-6-thread-50) BaseAsyncTask::RemoveTaskFromDB: Removed task db3e4ab4-6599-413a-a30d-5209a1eb3e78 from DataBase >2014-01-09 16:17:38,125 INFO [org.ovirt.engine.core.bll.CommandAsyncTask] (pool-6-thread-50) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity a8795666-ef0b-4314-834c-a43ec0c0a133 >2014-01-09 16:18:04,897 WARN [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ajp--127.0.0.1-8702-9) The message key AddVmFromSnapshot is missing from bundles/ExecutionMessages >2014-01-09 16:18:04,974 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (ajp--127.0.0.1-8702-9) Lock Acquired to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 16:18:05,189 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (ajp--127.0.0.1-8702-9) Running command: AddVmFromSnapshotCommand internal: false. Entities affected : ID: ac664a0e-2058-44ec-a240-be2adb31bd87 Type: VdsGroups, ID: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 Type: VM, ID: cbdefcb2-08ca-400f-84e0-1a6c7d28d763 Type: Storage, ID: cbdefcb2-08ca-400f-84e0-1a6c7d28d763 Type: Storage >2014-01-09 16:18:05,331 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (ajp--127.0.0.1-8702-9) Locking VM(id = e48c51ac-dc23-4992-a534-84ed5d1dbca5) with compensation. >2014-01-09 16:18:05,335 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (ajp--127.0.0.1-8702-9) START, SetVmStatusVDSCommand( vmId = e48c51ac-dc23-4992-a534-84ed5d1dbca5, status = ImageLocked), log id: 3a211680 >2014-01-09 16:18:05,339 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (ajp--127.0.0.1-8702-9) FINISH, SetVmStatusVDSCommand, log id: 3a211680 >2014-01-09 16:18:05,342 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (ajp--127.0.0.1-8702-9) Lock freed to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 16:18:05,363 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (ajp--127.0.0.1-8702-9) Running command: CopyImageGroupCommand internal: true. Entities affected : ID: cbdefcb2-08ca-400f-84e0-1a6c7d28d763 Type: Storage >2014-01-09 16:18:05,437 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp--127.0.0.1-8702-9) START, CopyImageVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, storageDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, imageGroupId = de474e2f-5669-47ac-9103-692fd87aad30, imageId = 40742666-bb4f-4741-9d1e-08dddb2dba27, dstImageGroupId = 4cd5c453-12b4-42f4-9a95-a3743e5afbd6, vmId = e48c51ac-dc23-4992-a534-84ed5d1dbca5, dstImageId = 4738d5e2-b276-4d20-990e-9189c7926e4a, imageDescription = , dstStorageDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, copyVolumeType = LeafVol, volumeFormat = RAW, preallocate = Preallocated, postZero = false, force = false), log id: 3ba7185a >2014-01-09 16:18:05,439 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp--127.0.0.1-8702-9) -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID >2014-01-09 16:18:05,440 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp--127.0.0.1-8702-9) -- copyImage parameters: > sdUUID=cbdefcb2-08ca-400f-84e0-1a6c7d28d763 > spUUID=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d > vmGUID=e48c51ac-dc23-4992-a534-84ed5d1dbca5 > srcImageGUID=de474e2f-5669-47ac-9103-692fd87aad30 > srcVolUUID=40742666-bb4f-4741-9d1e-08dddb2dba27 > dstImageGUID=4cd5c453-12b4-42f4-9a95-a3743e5afbd6 > dstVolUUID=4738d5e2-b276-4d20-990e-9189c7926e4a > descr= > dstSdUUID=cbdefcb2-08ca-400f-84e0-1a6c7d28d763 >2014-01-09 16:18:06,321 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp--127.0.0.1-8702-9) FINISH, CopyImageVDSCommand, return: 00000000-0000-0000-0000-000000000000, log id: 3ba7185a >2014-01-09 16:18:06,430 INFO [org.ovirt.engine.core.bll.CommandAsyncTask] (ajp--127.0.0.1-8702-9) CommandAsyncTask::Adding CommandMultiAsyncTasks object for command c472f044-8e48-4fe2-8cfd-d8f81b768ac0 >2014-01-09 16:18:06,431 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (ajp--127.0.0.1-8702-9) CommandMultiAsyncTasks::AttachTask: Attaching task ff40708c-b30c-48a0-a453-29e9447e33d6 to command c472f044-8e48-4fe2-8cfd-d8f81b768ac0. >2014-01-09 16:18:06,440 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (ajp--127.0.0.1-8702-9) Adding task ff40708c-b30c-48a0-a453-29e9447e33d6 (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-01-09 16:18:06,454 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (ajp--127.0.0.1-8702-9) Running command: CopyImageGroupCommand internal: true. Entities affected : ID: cbdefcb2-08ca-400f-84e0-1a6c7d28d763 Type: Storage >2014-01-09 16:18:06,475 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp--127.0.0.1-8702-9) START, CopyImageVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, storageDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, imageGroupId = d8f95faa-2402-4906-b6fe-d560edf56a4a, imageId = bd6325e5-934e-446d-a53e-1d5113188611, dstImageGroupId = 51fdfe7d-97eb-4c6b-affa-24eb2b9aa9d4, vmId = e48c51ac-dc23-4992-a534-84ed5d1dbca5, dstImageId = 1e72a129-913c-4893-bffd-0cf4f599a682, imageDescription = , dstStorageDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, copyVolumeType = LeafVol, volumeFormat = RAW, preallocate = Preallocated, postZero = false, force = false), log id: 1b735695 >2014-01-09 16:18:06,476 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp--127.0.0.1-8702-9) -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID >2014-01-09 16:18:06,477 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp--127.0.0.1-8702-9) -- copyImage parameters: > sdUUID=cbdefcb2-08ca-400f-84e0-1a6c7d28d763 > spUUID=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d > vmGUID=e48c51ac-dc23-4992-a534-84ed5d1dbca5 > srcImageGUID=d8f95faa-2402-4906-b6fe-d560edf56a4a > srcVolUUID=bd6325e5-934e-446d-a53e-1d5113188611 > dstImageGUID=51fdfe7d-97eb-4c6b-affa-24eb2b9aa9d4 > dstVolUUID=1e72a129-913c-4893-bffd-0cf4f599a682 > descr= > dstSdUUID=cbdefcb2-08ca-400f-84e0-1a6c7d28d763 >2014-01-09 16:18:08,117 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp--127.0.0.1-8702-9) FINISH, CopyImageVDSCommand, return: 00000000-0000-0000-0000-000000000000, log id: 1b735695 >2014-01-09 16:18:08,129 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (ajp--127.0.0.1-8702-9) CommandMultiAsyncTasks::AttachTask: Attaching task 3ba786e8-97a9-4d52-915c-7f9a7536eebc to command c472f044-8e48-4fe2-8cfd-d8f81b768ac0. >2014-01-09 16:18:08,143 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (ajp--127.0.0.1-8702-9) Adding task 3ba786e8-97a9-4d52-915c-7f9a7536eebc (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-01-09 16:18:08,223 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-9) Correlation ID: 2a2843ff, Job ID: f576e816-56fc-4dca-a9f4-0a283687fb36, Call Stack: null, Custom Event ID: -1, Message: VM syslog00-0 creation was initiated by ryan-a. >2014-01-09 16:18:08,223 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (ajp--127.0.0.1-8702-9) BaseAsyncTask::StartPollingTask: Starting to poll task ff40708c-b30c-48a0-a453-29e9447e33d6. >2014-01-09 16:18:08,224 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (ajp--127.0.0.1-8702-9) BaseAsyncTask::StartPollingTask: Starting to poll task 3ba786e8-97a9-4d52-915c-7f9a7536eebc. >2014-01-09 16:18:17,306 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (DefaultQuartzScheduler_Worker-31) Polling and updating Async Tasks: 4 tasks, 2 tasks to poll now >2014-01-09 16:18:17,319 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-31) SPMAsyncTask::PollTask: Polling task 3ba786e8-97a9-4d52-915c-7f9a7536eebc (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-01-09 16:18:17,320 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-31) SPMAsyncTask::PollTask: Polling task ff40708c-b30c-48a0-a453-29e9447e33d6 (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-01-09 16:18:17,321 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (DefaultQuartzScheduler_Worker-31) Finished polling Tasks, will poll again in 10 seconds. >2014-01-09 16:18:57,272 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (DefaultQuartzScheduler_Worker-74) Setting new tasks map. The map contains now 2 tasks >2014-01-09 16:18:57,359 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (DefaultQuartzScheduler_Worker-82) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-01-09 16:18:57,372 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (DefaultQuartzScheduler_Worker-82) Finished polling Tasks, will poll again in 10 seconds. >2014-01-09 16:22:05,421 INFO [org.ovirt.engine.core.bll.GetDiskAlignmentCommand] (ajp--127.0.0.1-8702-1) [4c6f21a8] Lock Acquired to object EngineLock [exclusiveLocks= key: 93aa4996-9cdd-4197-ab8d-18c269cbd9e7 value: DISK >, sharedLocks= ] >2014-01-09 16:22:05,445 INFO [org.ovirt.engine.core.bll.GetDiskAlignmentCommand] (pool-6-thread-49) [4c6f21a8] Running command: GetDiskAlignmentCommand internal: false. Entities affected : ID: 93aa4996-9cdd-4197-ab8d-18c269cbd9e7 Type: Disk, ID: 93aa4996-9cdd-4197-ab8d-18c269cbd9e7 Type: Disk >2014-01-09 16:22:05,453 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-49) [4c6f21a8] Correlation ID: 4c6f21a8, Job ID: 76a0b42f-3572-42a8-bd84-b1587a720137, Call Stack: null, Custom Event ID: -1, Message: Starting alignment scan of disk 'Apache_1_Disk1'. >2014-01-09 16:22:05,474 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDiskAlignmentVDSCommand] (pool-6-thread-49) [4c6f21a8] START, GetDiskAlignmentVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=00000000-0000-0000-0000-000000000000), log id: 4cc8bcd7 >2014-01-09 16:22:05,475 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDiskAlignmentVDSCommand] (pool-6-thread-49) [4c6f21a8] -- GetDiskAlignmentVDSCommand::executeVdsBrokerCommand: calling 'getDiskAlignment' >2014-01-09 16:22:05,476 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDiskAlignmentVDSCommand] (pool-6-thread-49) [4c6f21a8] -- getDiskAlignment parameters: > vmId=00000000-0000-0000-0000-000000000000 > driveSpecs={volumeID=7e14c768-bbf1-406c-bd74-b5d4ef5a0166, imageID=93aa4996-9cdd-4197-ab8d-18c269cbd9e7, domainID=cbdefcb2-08ca-400f-84e0-1a6c7d28d763, device=disk, poolID=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d} > >2014-01-09 16:22:26,992 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDiskAlignmentVDSCommand] (pool-6-thread-49) [4c6f21a8] Failed in GetDiskAlignmentVDS method, for vds: ovirt01; host: 192.168.100.21 >2014-01-09 16:22:26,993 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDiskAlignmentVDSCommand] (pool-6-thread-49) [4c6f21a8] Command GetDiskAlignmentVDS execution failed. Exception: ClassCastException: java.lang.Integer cannot be cast to java.util.Map >2014-01-09 16:22:26,994 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDiskAlignmentVDSCommand] (pool-6-thread-49) [4c6f21a8] FINISH, GetDiskAlignmentVDSCommand, log id: 4cc8bcd7 >2014-01-09 16:22:26,995 ERROR [org.ovirt.engine.core.bll.GetDiskAlignmentCommand] (pool-6-thread-49) [4c6f21a8] Command org.ovirt.engine.core.bll.GetDiskAlignmentCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.util.Map (Failed with error ENGINE and code 5001) >2014-01-09 16:22:27,006 INFO [org.ovirt.engine.core.bll.GetDiskAlignmentCommand] (pool-6-thread-49) [4c6f21a8] Command [id=8a815d31-2bd2-41b5-b63c-3383744e729d]: Compensating CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.businessentities.Image; snapshot: EntityStatusSnapshot [id=7e14c768-bbf1-406c-bd74-b5d4ef5a0166, status=OK]. >2014-01-09 16:22:27,011 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-49) [4c6f21a8] Correlation ID: 4c6f21a8, Job ID: 76a0b42f-3572-42a8-bd84-b1587a720137, Call Stack: null, Custom Event ID: -1, Message: Alignment scan of disk 'Apache_1_Disk1' failed. >2014-01-09 16:22:27,015 INFO [org.ovirt.engine.core.bll.GetDiskAlignmentCommand] (pool-6-thread-49) [4c6f21a8] Lock freed to object EngineLock [exclusiveLocks= key: 93aa4996-9cdd-4197-ab8d-18c269cbd9e7 value: DISK >, sharedLocks= ] >2014-01-09 16:28:54,899 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp--127.0.0.1-8702-6) Lock Acquired to object EngineLock [exclusiveLocks= key: Finance value: VM_NAME >, sharedLocks= ] >2014-01-09 16:28:54,909 WARN [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp--127.0.0.1-8702-6) CanDoAction of action UpdateVm failed. Reasons:VAR__ACTION__UPDATE,VAR__TYPE__VM,$device VirtIO-SCSI,VM_CANNOT_UPDATE_DEVICE_VM_NOT_DOWN >2014-01-09 16:28:54,910 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp--127.0.0.1-8702-6) Lock freed to object EngineLock [exclusiveLocks= key: Finance value: VM_NAME >, sharedLocks= ] >2014-01-09 16:30:28,249 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (ajp--127.0.0.1-8702-3) [eb39544] Lock Acquired to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 16:30:28,270 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (pool-6-thread-50) [eb39544] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: b6972001-2bd6-4cea-9c15-c046674754d4 Type: VM >2014-01-09 16:30:28,288 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-50) [eb39544] START, MigrateVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=b6972001-2bd6-4cea-9c15-c046674754d4, srcHost=192.168.100.22, dstVdsId=f2d7a8a9-e77e-4e9a-8031-23a0731407c8, dstHost=192.168.100.21:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 5b931d1d >2014-01-09 16:30:28,290 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-50) [eb39544] VdsBroker::migrate::Entered (vm_guid=b6972001-2bd6-4cea-9c15-c046674754d4, srcHost=192.168.100.22, dstHost=192.168.100.21:54321, method=online >2014-01-09 16:30:28,292 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-50) [eb39544] START, MigrateBrokerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=b6972001-2bd6-4cea-9c15-c046674754d4, srcHost=192.168.100.22, dstVdsId=f2d7a8a9-e77e-4e9a-8031-23a0731407c8, dstHost=192.168.100.21:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 3d0152ae >2014-01-09 16:30:28,305 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-50) [eb39544] FINISH, MigrateBrokerVDSCommand, log id: 3d0152ae >2014-01-09 16:30:28,309 INFO [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-50) [eb39544] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 5b931d1d >2014-01-09 16:30:28,316 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-50) [eb39544] Correlation ID: eb39544, Job ID: 8a7dbab3-6f10-44bb-bf86-64d1f1ffde59, Call Stack: null, Custom Event ID: -1, Message: Migration started (VM: DC-01, Source: ovirt02, Destination: ovirt01, User: ryan-a). >2014-01-09 16:30:29,162 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 16:30:32,192 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-51) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 16:30:35,214 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-55) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 16:30:38,234 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 16:30:41,258 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-65) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 16:30:44,285 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 16:30:47,312 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 16:30:50,334 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 16:30:53,361 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 is migrating to vds ovirt01 ignoring it in the refresh until migration is done >2014-01-09 16:30:56,422 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-74) START, FullListVdsCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01], vmIds=[b6972001-2bd6-4cea-9c15-c046674754d4]), log id: 264f126a >2014-01-09 16:30:56,433 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-74) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@44259ee, log id: 264f126a >2014-01-09 16:30:56,449 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-74) Correlation ID: eb39544, Job ID: 8a7dbab3-6f10-44bb-bf86-64d1f1ffde59, Call Stack: null, Custom Event ID: -1, Message: Migration completed (VM: DC-01, Source: ovirt02, Destination: ovirt01, Duration: 28 sec). >2014-01-09 16:30:56,451 INFO [org.ovirt.engine.core.bll.MigrateVmToServerCommand] (DefaultQuartzScheduler_Worker-74) Lock freed to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 16:30:58,654 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-88) START, DestroyVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=b6972001-2bd6-4cea-9c15-c046674754d4, force=false, secondsToWait=0, gracefully=false), log id: 227aed86 >2014-01-09 16:31:00,175 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-88) FINISH, DestroyVDSCommand, log id: 227aed86 >2014-01-09 16:31:00,184 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) RefreshVmList vm id b6972001-2bd6-4cea-9c15-c046674754d4 status = Down on vds ovirt02 ignoring it in the refresh until migration is done >2014-01-09 16:31:29,352 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand] (DefaultQuartzScheduler_Worker-28) Command ListVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:31:29,356 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-28) Failed to refresh VDS , vds = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 : ovirt02, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:31:30,528 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-21) Command HSMGetAllTasksStatusesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:31:34,366 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:31:34,368 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-39) Failed to refresh VDS , vds = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 : ovirt02, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:31:36,564 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-40) Command SpmStatusVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:31:36,566 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-40) Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 16:31:36,576 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-40) Correlation ID: 4b964584, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center CSEnt. Setting Data Center status to Non Responsive (On host ovirt02, Error: Network error during communication with the Host.). >2014-01-09 16:31:38,582 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-40) IrsBroker::Failed::GetStoragePoolInfoVDS due to: SocketTimeoutException: connect timed out >2014-01-09 16:31:39,378 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:31:39,380 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-45) Failed to refresh VDS , vds = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 : ovirt02, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:31:40,589 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-40) Command SpmStatusVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:31:40,590 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-40) Irs placed on server bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 failed. Proceed Failover >2014-01-09 16:31:40,610 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-40) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:31:40,615 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-40) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:31:40,618 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-40) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:31:40,620 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-40) START, SpmStopVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 67ebfd78 >2014-01-09 16:31:40,621 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-40) SpmStopVDSCommand:: vds ovirt02 is in Connecting status - not performing spm stop, pool id 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 16:31:40,622 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-40) FINISH, SpmStopVDSCommand, log id: 67ebfd78 >2014-01-09 16:31:40,623 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-40) spm stop on spm failed, stopping spm selection! >2014-01-09 16:31:40,638 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:31:40,640 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:31:40,642 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:31:40,644 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-46) START, SpmStopVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 162d6e1c >2014-01-09 16:31:40,644 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-46) SpmStopVDSCommand:: vds ovirt02 is in Connecting status - not performing spm stop, pool id 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 16:31:40,645 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-46) FINISH, SpmStopVDSCommand, log id: 162d6e1c >2014-01-09 16:31:40,645 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) spm stop on spm failed, stopping spm selection! >2014-01-09 16:31:44,389 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:31:44,391 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-49) Failed to refresh VDS , vds = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 : ovirt02, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:31:49,400 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:31:49,402 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-47) Failed to refresh VDS , vds = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 : ovirt02, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:31:50,638 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-54) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:31:50,640 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-54) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:31:50,642 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-54) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:31:50,644 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-54) START, SpmStopVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: f3a4d80 >2014-01-09 16:31:50,645 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-54) SpmStopVDSCommand:: vds ovirt02 is in Connecting status - not performing spm stop, pool id 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 16:31:50,646 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-54) FINISH, SpmStopVDSCommand, log id: f3a4d80 >2014-01-09 16:31:50,646 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-54) spm stop on spm failed, stopping spm selection! >2014-01-09 16:31:50,658 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-56) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:31:50,660 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-56) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:31:50,663 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-56) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:31:50,664 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-56) START, SpmStopVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 1690415d >2014-01-09 16:31:50,665 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-56) SpmStopVDSCommand:: vds ovirt02 is in Connecting status - not performing spm stop, pool id 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 16:31:50,666 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-56) FINISH, SpmStopVDSCommand, log id: 1690415d >2014-01-09 16:31:50,667 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-56) spm stop on spm failed, stopping spm selection! >2014-01-09 16:31:54,411 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:31:54,413 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) Failed to refresh VDS , vds = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 : ovirt02, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:31:59,422 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:31:59,424 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-65) Failed to refresh VDS , vds = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 : ovirt02, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:32:00,660 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:32:00,662 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:32:00,665 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:32:00,667 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-73) START, SpmStopVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 50cdda97 >2014-01-09 16:32:00,668 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-73) SpmStopVDSCommand:: vds ovirt02 is in Connecting status - not performing spm stop, pool id 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 16:32:00,669 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-73) FINISH, SpmStopVDSCommand, log id: 50cdda97 >2014-01-09 16:32:00,669 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) spm stop on spm failed, stopping spm selection! >2014-01-09 16:32:00,681 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-68) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:32:00,683 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-68) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:32:00,685 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-68) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:32:00,687 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-68) START, SpmStopVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 686d512c >2014-01-09 16:32:00,688 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-68) SpmStopVDSCommand:: vds ovirt02 is in Connecting status - not performing spm stop, pool id 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 16:32:00,689 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-68) FINISH, SpmStopVDSCommand, log id: 686d512c >2014-01-09 16:32:00,689 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-68) spm stop on spm failed, stopping spm selection! >2014-01-09 16:32:04,433 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:32:04,435 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-75) Failed to refresh VDS , vds = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 : ovirt02, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:32:09,444 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-77) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:32:09,445 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-77) Failed to refresh VDS , vds = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 : ovirt02, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:32:10,683 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-81) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:32:10,685 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-81) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:32:10,687 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-81) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:32:10,689 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-81) START, SpmStopVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 7daba8b8 >2014-01-09 16:32:10,690 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-81) SpmStopVDSCommand:: vds ovirt02 is in Connecting status - not performing spm stop, pool id 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 16:32:10,691 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-81) FINISH, SpmStopVDSCommand, log id: 7daba8b8 >2014-01-09 16:32:10,691 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-81) spm stop on spm failed, stopping spm selection! >2014-01-09 16:32:10,703 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-83) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:32:10,705 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-83) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:32:10,708 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-83) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:32:10,709 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-83) START, SpmStopVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 551486fb >2014-01-09 16:32:10,710 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-83) SpmStopVDSCommand:: vds ovirt02 is in Connecting status - not performing spm stop, pool id 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 16:32:10,711 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-83) FINISH, SpmStopVDSCommand, log id: 551486fb >2014-01-09 16:32:10,712 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-83) spm stop on spm failed, stopping spm selection! >2014-01-09 16:32:14,454 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:32:14,456 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-74) Failed to refresh VDS , vds = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 : ovirt02, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:32:19,465 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:32:19,467 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-85) Failed to refresh VDS , vds = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 : ovirt02, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:32:20,705 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-92) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:32:20,707 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-92) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:32:20,709 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-92) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:32:20,711 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-92) START, SpmStopVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 7ef205a5 >2014-01-09 16:32:20,712 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-92) SpmStopVDSCommand:: vds ovirt02 is in Connecting status - not performing spm stop, pool id 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 16:32:20,713 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-92) FINISH, SpmStopVDSCommand, log id: 7ef205a5 >2014-01-09 16:32:20,713 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-92) spm stop on spm failed, stopping spm selection! >2014-01-09 16:32:20,724 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-90) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:32:20,726 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-90) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:32:20,728 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-90) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:32:20,730 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-90) START, SpmStopVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 613d4986 >2014-01-09 16:32:20,731 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-90) SpmStopVDSCommand:: vds ovirt02 is in Connecting status - not performing spm stop, pool id 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 16:32:20,732 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-90) FINISH, SpmStopVDSCommand, log id: 613d4986 >2014-01-09 16:32:20,732 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-90) spm stop on spm failed, stopping spm selection! >2014-01-09 16:32:24,477 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:32:24,479 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-97) Failed to refresh VDS , vds = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 : ovirt02, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:32:29,488 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:32:29,490 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-100) Failed to refresh VDS , vds = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 : ovirt02, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:32:30,727 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-7) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:32:30,730 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-7) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:32:30,732 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-7) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:32:30,734 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-7) START, SpmStopVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 68f3dc61 >2014-01-09 16:32:30,735 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-7) SpmStopVDSCommand:: vds ovirt02 is in Connecting status - not performing spm stop, pool id 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 16:32:30,736 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-7) FINISH, SpmStopVDSCommand, log id: 68f3dc61 >2014-01-09 16:32:30,737 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-7) spm stop on spm failed, stopping spm selection! >2014-01-09 16:32:30,748 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:32:30,750 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:32:30,753 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:32:30,756 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-8) START, SpmStopVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 58b001cb >2014-01-09 16:32:30,757 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-8) SpmStopVDSCommand:: vds ovirt02 is in Connecting status - not performing spm stop, pool id 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 16:32:30,757 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-8) FINISH, SpmStopVDSCommand, log id: 58b001cb >2014-01-09 16:32:30,758 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) spm stop on spm failed, stopping spm selection! >2014-01-09 16:32:34,499 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:32:34,501 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-12) Failed to refresh VDS , vds = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 : ovirt02, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:32:39,510 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:32:39,512 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-17) Failed to refresh VDS , vds = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 : ovirt02, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:32:40,751 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-20) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:32:40,753 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-20) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:32:40,755 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-20) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:32:40,756 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-20) START, SpmStopVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 7025ab3a >2014-01-09 16:32:40,757 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-20) SpmStopVDSCommand:: vds ovirt02 is in Connecting status - not performing spm stop, pool id 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 16:32:40,758 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-20) FINISH, SpmStopVDSCommand, log id: 7025ab3a >2014-01-09 16:32:40,759 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-20) spm stop on spm failed, stopping spm selection! >2014-01-09 16:32:40,769 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-22) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:32:40,771 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-22) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:32:40,773 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-22) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:32:40,775 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-22) START, SpmStopVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 162f5bc5 >2014-01-09 16:32:40,775 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-22) SpmStopVDSCommand:: vds ovirt02 is in Connecting status - not performing spm stop, pool id 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 16:32:40,776 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-22) FINISH, SpmStopVDSCommand, log id: 162f5bc5 >2014-01-09 16:32:40,777 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-22) spm stop on spm failed, stopping spm selection! >2014-01-09 16:32:44,521 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:32:44,523 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Failed to refresh VDS , vds = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 : ovirt02, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:32:49,533 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:32:49,534 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-19) Server failed to respond, vds_id = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds_name = ovirt02, vm_count = 3, spm_status = SPM, non-responsive_timeout (seconds) = 81, error = java.net.SocketTimeoutException: connect timed out >2014-01-09 16:32:49,540 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-19) Correlation ID: null, Call Stack: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:102) > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:28) > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:536) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:492) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.Refresh(VdsUpdateRunTimeInfo.java:343) > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:237) > at sun.reflect.GeneratedMethodAccessor50.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > at java.net.Socket.connect(Socket.java:579) > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) > at com.sun.proxy.$Proxy52.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor228.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:225) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:744) >, Custom Event ID: -1, Message: Host ovirt02 is non responsive. >2014-01-09 16:32:49,554 INFO [org.ovirt.engine.core.bll.VdsEventListener] (pool-6-thread-49) ResourceManager::vdsNotResponding entered for Host bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, 192.168.100.22 >2014-01-09 16:32:49,568 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (pool-6-thread-49) Running command: SshSoftFencingCommand internal: true. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 16:32:50,532 ERROR [org.ovirt.engine.core.bll.SshSoftFencingCommand] (pool-6-thread-49) SSH connection to host 192.168.100.22 failed: org.apache.sshd.common.RuntimeSshException: Failed to get the session.: org.apache.sshd.common.RuntimeSshException: Failed to get the session. > at org.apache.sshd.client.future.DefaultConnectFuture.getSession(DefaultConnectFuture.java:43) > at org.ovirt.engine.core.utils.ssh.SSHClient.connect(SSHClient.java:288) [utils.jar:] > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:59) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeSshSoftFencingCommand(SshSoftFencingCommand.java:73) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeCommand(SshSoftFencingCommand.java:43) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1135) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1220) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1896) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1240) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:366) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:416) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:395) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:636) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view10.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.VdsEventListener$1.run(VdsEventListener.java:132) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:71) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_45] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_45] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_45] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_45] > at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_45] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735) [rt.jar:1.7.0_45] > at org.apache.mina.transport.socket.nio.NioSocketConnector.finishConnect(NioSocketConnector.java:223) [mina-core.jar:] > at org.apache.mina.transport.socket.nio.NioSocketConnector.finishConnect(NioSocketConnector.java:45) [mina-core.jar:] > at org.apache.mina.core.polling.AbstractPollingIoConnector.processConnections(AbstractPollingIoConnector.java:427) [mina-core.jar:] > at org.apache.mina.core.polling.AbstractPollingIoConnector.access$500(AbstractPollingIoConnector.java:62) [mina-core.jar:] > at org.apache.mina.core.polling.AbstractPollingIoConnector$Connector.run(AbstractPollingIoConnector.java:476) [mina-core.jar:] > at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64) [mina-core.jar:] > ... 3 more > >2014-01-09 16:32:50,598 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-49) START, SetVmStatusVDSCommand( vmId = 33f88d50-30eb-45c7-9850-733da17ed7a4, status = Unknown), log id: 574a8587 >2014-01-09 16:32:50,601 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-49) FINISH, SetVmStatusVDSCommand, log id: 574a8587 >2014-01-09 16:32:50,607 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-49) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM DC-02 was set to the Unknown status. >2014-01-09 16:32:50,607 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-49) START, SetVmStatusVDSCommand( vmId = 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, status = Unknown), log id: 34a29aad >2014-01-09 16:32:50,610 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-49) FINISH, SetVmStatusVDSCommand, log id: 34a29aad >2014-01-09 16:32:50,615 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-49) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM VETS was set to the Unknown status. >2014-01-09 16:32:50,616 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-49) START, SetVmStatusVDSCommand( vmId = 872c3dea-6984-48fa-99c3-354fd367c3f9, status = Unknown), log id: 6ae00413 >2014-01-09 16:32:50,618 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-49) FINISH, SetVmStatusVDSCommand, log id: 6ae00413 >2014-01-09 16:32:50,623 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-49) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM External_Email was set to the Unknown status. >2014-01-09 16:32:50,624 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (pool-6-thread-49) Fence failed on vds ovirt02 which is spm of pool 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d - moving pool to non operational >2014-01-09 16:32:50,626 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (pool-6-thread-49) Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 16:32:50,636 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-49) Correlation ID: 52cf273d, Call Stack: null, Custom Event ID: -1, Message: Fencing failed on Storage Pool Manager ovirt02 for Data Center CSEnt. Setting status to Non-Operational. >2014-01-09 16:32:50,637 ERROR [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (pool-6-thread-49) Failed to run Fence script on vds:ovirt02, VMs moved to UnKnown instead. >2014-01-09 16:32:50,643 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-49) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host ovirt02 became non responsive. It has no power management configured. Please check the host status, manually reboot it, and click "Confirm Host Has Been Rebooted" >2014-01-09 16:32:50,644 WARN [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (pool-6-thread-49) CanDoAction of action VdsNotRespondingTreatment failed. Reasons:VDS_FENCE_DISABLED >2014-01-09 16:32:50,782 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-32) Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 16:32:50,790 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-32) Correlation ID: 29763ec9, Call Stack: null, Custom Event ID: -1, Message: Data Center is being initialized, please wait for initialization to complete. >2014-01-09 16:32:50,801 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-32) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:32:50,803 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-32) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:32:50,805 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-32) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:32:50,806 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-32) spm vds is non responsive, stopping spm selection. >2014-01-09 16:32:54,563 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:32:59,574 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:33:00,772 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-40) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:33:00,774 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-40) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:33:00,776 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-40) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:33:00,777 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-40) spm vds is non responsive, stopping spm selection. >2014-01-09 16:33:00,817 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:33:00,819 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:33:00,821 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:33:00,822 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) spm vds is non responsive, stopping spm selection. >2014-01-09 16:33:04,584 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:33:09,594 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:33:10,790 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-54) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:33:10,792 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-54) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:33:10,795 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-54) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:33:10,795 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-54) spm vds is non responsive, stopping spm selection. >2014-01-09 16:33:10,832 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-56) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:33:10,834 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-56) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:33:10,836 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-56) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:33:10,837 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-56) spm vds is non responsive, stopping spm selection. >2014-01-09 16:33:14,603 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:33:19,612 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:33:20,809 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-69) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:33:20,810 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-69) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:33:20,813 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-69) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:33:20,813 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-69) spm vds is non responsive, stopping spm selection. >2014-01-09 16:33:20,848 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-66) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:33:20,849 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-66) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:33:20,851 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-66) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:33:20,852 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-66) spm vds is non responsive, stopping spm selection. >2014-01-09 16:33:24,622 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:33:29,631 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:33:30,828 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-74) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:33:30,830 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-74) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:33:30,832 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-74) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:33:30,833 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-74) spm vds is non responsive, stopping spm selection. >2014-01-09 16:33:30,863 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-89) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:33:30,865 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-89) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:33:30,867 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-89) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:33:30,868 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-89) spm vds is non responsive, stopping spm selection. >2014-01-09 16:33:34,642 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:33:39,652 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-88) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:33:40,847 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-94) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:33:40,849 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-94) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:33:40,851 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-94) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:33:40,852 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-94) spm vds is non responsive, stopping spm selection. >2014-01-09 16:33:40,879 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-97) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:33:40,881 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-97) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:33:40,883 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-97) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:33:40,883 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-97) spm vds is non responsive, stopping spm selection. >2014-01-09 16:33:44,662 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:33:49,672 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:33:50,867 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-3) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:33:50,868 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-3) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:33:50,871 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-3) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:33:50,871 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-3) spm vds is non responsive, stopping spm selection. >2014-01-09 16:33:50,895 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-5) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:33:50,897 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-5) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:33:50,899 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-5) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:33:50,900 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-5) spm vds is non responsive, stopping spm selection. >2014-01-09 16:33:54,682 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:33:59,693 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:34:00,884 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-20) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:34:00,886 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-20) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:34:00,889 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-20) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:34:00,889 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-20) spm vds is non responsive, stopping spm selection. >2014-01-09 16:34:00,910 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-22) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:34:00,912 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-22) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:34:00,914 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-22) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:34:00,915 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-22) spm vds is non responsive, stopping spm selection. >2014-01-09 16:34:04,703 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-27) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:34:09,711 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-29) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:34:10,903 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-28) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:34:10,904 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-28) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:34:10,906 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-28) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:34:10,907 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-28) spm vds is non responsive, stopping spm selection. >2014-01-09 16:34:10,926 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-32) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:34:10,928 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-32) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:34:10,930 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-32) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:34:10,931 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-32) spm vds is non responsive, stopping spm selection. >2014-01-09 16:34:14,722 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-21) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:34:19,735 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:34:20,921 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-13) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:34:20,923 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-13) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:34:20,925 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-13) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:34:20,926 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-13) spm vds is non responsive, stopping spm selection. >2014-01-09 16:34:20,942 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:34:20,944 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:34:20,946 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:34:20,947 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) spm vds is non responsive, stopping spm selection. >2014-01-09 16:34:24,747 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:34:29,758 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:34:30,940 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-60) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:34:30,941 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-60) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:34:30,944 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-60) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:34:30,944 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-60) spm vds is non responsive, stopping spm selection. >2014-01-09 16:34:30,955 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:34:30,957 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:34:30,959 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:34:30,960 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) spm vds is non responsive, stopping spm selection. >2014-01-09 16:34:34,770 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-57) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:34:39,781 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:34:40,958 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-68) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:34:40,959 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-68) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:34:40,962 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-68) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:34:40,962 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-68) spm vds is non responsive, stopping spm selection. >2014-01-09 16:34:40,974 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:34:40,976 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:34:40,978 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:34:40,979 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) spm vds is non responsive, stopping spm selection. >2014-01-09 16:34:44,794 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:34:49,806 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-77) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:34:50,976 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-83) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:34:50,978 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-83) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:34:50,980 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-83) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:34:50,981 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-83) spm vds is non responsive, stopping spm selection. >2014-01-09 16:34:50,991 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-63) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:34:50,993 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-63) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:34:50,995 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-63) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:34:50,996 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-63) spm vds is non responsive, stopping spm selection. >2014-01-09 16:34:54,818 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:34:59,830 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:35:00,993 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-97) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:35:00,994 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-97) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:35:00,997 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-97) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:35:00,998 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-97) spm vds is non responsive, stopping spm selection. >2014-01-09 16:35:01,011 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-91) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:35:01,014 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-91) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:35:01,016 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-91) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:35:01,017 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-91) spm vds is non responsive, stopping spm selection. >2014-01-09 16:35:04,842 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:35:09,853 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:35:11,050 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-5) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:35:11,052 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-5) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:35:11,054 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-5) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:35:11,054 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-5) spm vds is non responsive, stopping spm selection. >2014-01-09 16:35:11,066 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-6) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:35:11,067 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-6) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:35:11,069 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-6) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:35:11,070 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-6) spm vds is non responsive, stopping spm selection. >2014-01-09 16:35:14,865 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:35:19,876 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:35:21,102 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-10) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:35:21,103 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-10) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:35:21,105 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-10) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:35:21,106 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-10) spm vds is non responsive, stopping spm selection. >2014-01-09 16:35:21,118 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-15) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:35:21,119 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-15) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:35:21,121 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-15) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:35:21,122 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-15) spm vds is non responsive, stopping spm selection. >2014-01-09 16:35:24,887 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:35:29,897 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:35:31,161 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-38) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:35:31,162 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-38) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:35:31,165 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-38) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:35:31,166 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-38) spm vds is non responsive, stopping spm selection. >2014-01-09 16:35:31,185 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:35:31,186 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:35:31,189 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:35:31,189 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) spm vds is non responsive, stopping spm selection. >2014-01-09 16:35:34,909 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:35:39,921 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:35:41,179 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-49) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:35:41,181 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-49) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:35:41,183 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-49) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:35:41,184 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-49) spm vds is non responsive, stopping spm selection. >2014-01-09 16:35:41,225 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-42) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:35:41,227 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-42) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:35:41,229 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-42) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:35:41,230 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-42) spm vds is non responsive, stopping spm selection. >2014-01-09 16:35:44,933 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:35:49,944 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:35:51,197 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:35:51,198 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:35:51,201 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:35:51,201 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) spm vds is non responsive, stopping spm selection. >2014-01-09 16:35:51,241 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-55) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:35:51,242 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-55) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:35:51,244 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-55) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:35:51,245 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-55) spm vds is non responsive, stopping spm selection. >2014-01-09 16:35:54,955 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:35:59,968 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:36:01,216 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-68) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:36:01,217 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-68) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:36:01,220 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-68) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:36:01,220 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-68) spm vds is non responsive, stopping spm selection. >2014-01-09 16:36:01,255 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:36:01,257 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:36:01,259 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:36:01,260 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) spm vds is non responsive, stopping spm selection. >2014-01-09 16:36:04,981 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:36:09,993 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-81) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:36:11,234 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-63) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:36:11,236 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-63) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:36:11,238 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-63) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:36:11,238 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-63) spm vds is non responsive, stopping spm selection. >2014-01-09 16:36:11,270 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-86) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:36:11,272 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-86) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:36:11,274 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-86) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:36:11,274 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-86) spm vds is non responsive, stopping spm selection. >2014-01-09 16:36:15,004 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-87) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:36:20,016 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:36:21,251 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-82) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:36:21,253 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-82) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:36:21,255 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-82) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:36:21,256 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-82) spm vds is non responsive, stopping spm selection. >2014-01-09 16:36:21,285 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-88) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:36:21,287 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-88) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:36:21,289 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-88) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:36:21,290 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-88) spm vds is non responsive, stopping spm selection. >2014-01-09 16:36:25,028 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:36:30,040 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:36:31,277 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:36:31,279 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:36:31,281 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:36:31,282 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) spm vds is non responsive, stopping spm selection. >2014-01-09 16:36:31,302 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-14) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:36:31,304 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-14) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:36:31,306 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-14) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:36:31,307 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-14) spm vds is non responsive, stopping spm selection. >2014-01-09 16:36:35,052 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:36:40,064 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:36:41,295 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-24) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:36:41,296 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-24) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:36:41,299 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-24) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:36:41,299 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-24) spm vds is non responsive, stopping spm selection. >2014-01-09 16:36:41,319 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:36:41,321 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:36:41,323 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:36:41,324 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) spm vds is non responsive, stopping spm selection. >2014-01-09 16:36:45,077 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:36:50,088 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:36:51,313 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-38) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:36:51,314 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-38) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:36:51,316 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-38) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:36:51,317 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-38) spm vds is non responsive, stopping spm selection. >2014-01-09 16:36:51,334 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:36:51,336 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:36:51,339 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:36:51,339 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) spm vds is non responsive, stopping spm selection. >2014-01-09 16:36:55,101 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-41) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:37:00,113 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:37:01,331 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:37:01,333 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:37:01,335 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:37:01,336 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) spm vds is non responsive, stopping spm selection. >2014-01-09 16:37:01,351 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-44) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:37:01,353 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-44) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:37:01,355 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-44) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:37:01,356 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-44) spm vds is non responsive, stopping spm selection. >2014-01-09 16:37:05,126 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:37:10,138 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:37:11,349 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-59) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:37:11,350 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-59) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:37:11,352 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-59) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:37:11,353 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-59) spm vds is non responsive, stopping spm selection. >2014-01-09 16:37:11,364 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:37:11,366 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:37:11,368 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:37:11,369 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) spm vds is non responsive, stopping spm selection. >2014-01-09 16:37:15,149 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:37:20,160 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:37:21,367 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:37:21,368 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:37:21,371 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:37:21,371 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) spm vds is non responsive, stopping spm selection. >2014-01-09 16:37:21,383 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-75) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:37:21,384 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-75) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:37:21,386 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-75) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:37:21,387 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-75) spm vds is non responsive, stopping spm selection. >2014-01-09 16:37:25,171 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:37:30,184 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:37:31,386 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:37:31,388 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:37:31,390 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:37:31,391 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) spm vds is non responsive, stopping spm selection. >2014-01-09 16:37:31,403 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-92) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:37:31,405 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-92) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:37:31,408 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-92) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:37:31,408 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-92) spm vds is non responsive, stopping spm selection. >2014-01-09 16:37:35,196 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:37:40,207 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:37:41,405 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:37:41,407 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:37:41,410 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:37:41,410 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) spm vds is non responsive, stopping spm selection. >2014-01-09 16:37:41,422 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-4) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:37:41,423 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-4) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:37:41,426 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-4) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:37:41,426 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-4) spm vds is non responsive, stopping spm selection. >2014-01-09 16:37:45,220 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:37:50,232 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:37:51,424 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:37:51,426 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:37:51,428 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:37:51,428 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) spm vds is non responsive, stopping spm selection. >2014-01-09 16:37:51,439 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:37:51,440 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:37:51,442 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:37:51,443 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) spm vds is non responsive, stopping spm selection. >2014-01-09 16:37:55,244 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:38:00,257 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:38:01,443 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-24) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:38:01,445 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-24) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:38:01,447 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-24) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:38:01,448 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-24) spm vds is non responsive, stopping spm selection. >2014-01-09 16:38:01,459 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:38:01,461 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:38:01,463 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:38:01,464 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) spm vds is non responsive, stopping spm selection. >2014-01-09 16:38:05,269 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-29) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:38:10,281 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:38:11,460 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:38:11,462 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:38:11,464 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:38:11,465 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) spm vds is non responsive, stopping spm selection. >2014-01-09 16:38:11,476 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-37) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:38:11,478 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-37) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:38:11,480 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-37) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:38:11,480 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-37) spm vds is non responsive, stopping spm selection. >2014-01-09 16:38:15,293 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:38:20,305 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:38:21,478 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:38:21,480 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:38:21,482 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:38:21,483 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) spm vds is non responsive, stopping spm selection. >2014-01-09 16:38:21,493 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:38:21,494 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:38:21,496 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:38:21,497 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) spm vds is non responsive, stopping spm selection. >2014-01-09 16:38:25,319 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:38:30,331 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:38:31,497 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-66) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:38:31,499 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-66) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:38:31,501 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-66) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:38:31,501 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-66) spm vds is non responsive, stopping spm selection. >2014-01-09 16:38:31,513 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-57) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:38:31,514 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-57) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:38:31,517 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-57) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:38:31,517 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-57) spm vds is non responsive, stopping spm selection. >2014-01-09 16:38:35,344 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:38:40,356 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:38:41,517 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:38:41,519 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:38:41,521 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:38:41,522 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) spm vds is non responsive, stopping spm selection. >2014-01-09 16:38:41,532 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:38:41,534 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:38:41,536 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:38:41,537 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) spm vds is non responsive, stopping spm selection. >2014-01-09 16:38:45,369 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:38:50,379 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:38:51,534 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-85) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:38:51,536 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-85) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:38:51,538 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-85) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:38:51,538 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-85) spm vds is non responsive, stopping spm selection. >2014-01-09 16:38:51,548 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-83) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:38:51,550 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-83) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:38:51,552 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-83) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:38:51,553 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-83) spm vds is non responsive, stopping spm selection. >2014-01-09 16:38:55,391 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:39:00,402 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:39:01,558 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-98) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:39:01,559 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-98) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:39:01,562 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-98) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:39:01,562 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-98) spm vds is non responsive, stopping spm selection. >2014-01-09 16:39:01,574 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:39:01,576 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:39:01,578 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:39:01,579 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) spm vds is non responsive, stopping spm selection. >2014-01-09 16:39:05,415 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:39:10,427 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:39:11,574 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:39:11,576 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:39:11,578 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:39:11,578 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) spm vds is non responsive, stopping spm selection. >2014-01-09 16:39:11,589 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:39:11,590 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:39:11,592 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:39:11,593 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) spm vds is non responsive, stopping spm selection. >2014-01-09 16:39:15,438 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:39:20,448 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:39:21,590 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-20) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:39:21,592 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-20) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:39:21,594 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-20) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:39:21,595 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-20) spm vds is non responsive, stopping spm selection. >2014-01-09 16:39:21,605 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-15) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:39:21,607 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-15) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:39:21,609 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-15) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:39:21,609 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-15) spm vds is non responsive, stopping spm selection. >2014-01-09 16:39:25,460 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:39:30,472 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:39:31,608 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-39) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:39:31,609 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-39) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:39:31,612 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-39) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:39:31,612 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-39) spm vds is non responsive, stopping spm selection. >2014-01-09 16:39:31,624 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:39:31,625 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:39:31,627 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:39:31,628 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) spm vds is non responsive, stopping spm selection. >2014-01-09 16:39:35,484 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:39:40,495 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:39:41,625 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-44) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:39:41,627 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-44) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:39:41,629 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-44) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:39:41,630 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-44) spm vds is non responsive, stopping spm selection. >2014-01-09 16:39:41,640 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-51) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:39:41,642 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-51) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:39:41,644 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-51) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:39:41,645 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-51) spm vds is non responsive, stopping spm selection. >2014-01-09 16:39:45,507 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:39:50,519 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:39:51,642 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-58) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:39:51,643 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-58) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:39:51,646 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-58) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:39:51,646 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-58) spm vds is non responsive, stopping spm selection. >2014-01-09 16:39:51,657 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:39:51,659 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:39:51,661 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:39:51,661 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) spm vds is non responsive, stopping spm selection. >2014-01-09 16:39:55,531 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:40:00,543 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:40:01,660 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:40:01,662 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:40:01,664 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:40:01,665 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) spm vds is non responsive, stopping spm selection. >2014-01-09 16:40:01,675 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:40:01,677 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:40:01,679 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:40:01,680 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) spm vds is non responsive, stopping spm selection. >2014-01-09 16:40:05,555 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-81) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:40:10,567 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-89) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:40:11,677 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-85) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:40:11,679 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-85) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:40:11,681 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-85) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:40:11,681 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-85) spm vds is non responsive, stopping spm selection. >2014-01-09 16:40:11,692 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-83) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:40:11,693 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-83) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:40:11,695 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-83) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:40:11,696 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-83) spm vds is non responsive, stopping spm selection. >2014-01-09 16:40:15,579 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:40:20,589 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:40:21,694 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:40:21,695 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:40:21,698 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:40:21,698 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) spm vds is non responsive, stopping spm selection. >2014-01-09 16:40:21,709 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-98) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:40:21,711 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-98) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:40:21,713 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-98) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:40:21,713 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-98) spm vds is non responsive, stopping spm selection. >2014-01-09 16:40:25,602 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:40:30,613 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:40:31,711 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-17) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:40:31,713 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-17) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:40:31,715 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-17) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:40:31,716 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-17) spm vds is non responsive, stopping spm selection. >2014-01-09 16:40:31,726 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:40:31,728 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:40:31,730 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:40:31,730 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) spm vds is non responsive, stopping spm selection. >2014-01-09 16:40:35,624 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:40:40,636 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:40:41,729 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-31) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:40:41,730 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-31) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:40:41,732 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-31) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:40:41,733 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-31) spm vds is non responsive, stopping spm selection. >2014-01-09 16:40:41,744 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:40:41,746 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:40:41,748 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:40:41,749 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) spm vds is non responsive, stopping spm selection. >2014-01-09 16:40:45,649 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:40:50,662 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:40:51,745 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-32) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:40:51,746 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-32) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:40:51,749 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-32) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:40:51,749 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-32) spm vds is non responsive, stopping spm selection. >2014-01-09 16:40:51,760 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-39) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:40:51,761 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-39) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:40:51,764 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-39) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:40:51,764 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-39) spm vds is non responsive, stopping spm selection. >2014-01-09 16:40:55,674 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-41) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:41:00,686 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:41:01,762 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-53) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:41:01,764 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-53) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:41:01,767 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-53) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:41:01,767 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-53) spm vds is non responsive, stopping spm selection. >2014-01-09 16:41:01,778 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-51) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:41:01,780 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-51) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:41:01,783 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-51) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:41:01,784 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-51) spm vds is non responsive, stopping spm selection. >2014-01-09 16:41:05,698 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:41:10,710 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:41:11,821 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-58) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:41:11,822 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-58) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:41:11,825 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-58) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:41:11,825 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-58) spm vds is non responsive, stopping spm selection. >2014-01-09 16:41:11,837 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:41:11,839 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:41:11,841 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:41:11,842 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) spm vds is non responsive, stopping spm selection. >2014-01-09 16:41:15,722 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:41:20,733 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:41:21,841 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-76) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:41:21,843 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-76) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:41:21,845 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-76) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:41:21,846 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-76) spm vds is non responsive, stopping spm selection. >2014-01-09 16:41:21,857 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-69) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:41:21,858 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-69) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:41:21,861 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-69) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:41:21,861 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-69) spm vds is non responsive, stopping spm selection. >2014-01-09 16:41:25,745 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:41:30,756 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-89) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:41:31,858 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-70) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:41:31,860 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-70) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:41:31,863 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-70) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:41:31,863 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-70) spm vds is non responsive, stopping spm selection. >2014-01-09 16:41:31,874 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-82) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:41:31,876 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-82) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:41:31,878 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-82) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:41:31,878 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-82) spm vds is non responsive, stopping spm selection. >2014-01-09 16:41:35,770 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:41:40,783 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:41:41,877 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-1) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:41:41,878 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-1) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:41:41,880 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-1) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:41:41,881 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-1) spm vds is non responsive, stopping spm selection. >2014-01-09 16:41:41,890 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:41:41,891 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:41:41,894 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:41:41,894 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) spm vds is non responsive, stopping spm selection. >2014-01-09 16:41:45,793 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:41:50,805 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:41:51,893 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-17) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:41:51,896 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-17) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:41:51,898 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-17) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:41:51,899 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-17) spm vds is non responsive, stopping spm selection. >2014-01-09 16:41:51,909 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:41:51,910 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:41:51,912 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:41:51,913 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) spm vds is non responsive, stopping spm selection. >2014-01-09 16:41:55,815 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:42:00,827 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-27) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:42:01,911 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-24) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:42:01,913 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-24) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:42:01,915 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-24) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:42:01,916 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-24) spm vds is non responsive, stopping spm selection. >2014-01-09 16:42:01,927 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-29) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:42:01,929 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-29) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:42:01,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-29) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:42:01,932 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-29) spm vds is non responsive, stopping spm selection. >2014-01-09 16:42:05,840 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:42:10,850 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:42:11,929 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-39) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:42:11,930 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-39) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:42:11,933 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-39) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:42:11,933 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-39) spm vds is non responsive, stopping spm selection. >2014-01-09 16:42:11,944 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-36) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:42:11,946 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-36) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:42:11,948 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-36) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:42:11,949 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-36) spm vds is non responsive, stopping spm selection. >2014-01-09 16:42:15,862 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:42:20,874 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:42:21,946 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-53) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:42:21,947 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-53) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:42:21,949 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-53) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:42:21,950 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-53) spm vds is non responsive, stopping spm selection. >2014-01-09 16:42:21,961 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-51) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:42:21,963 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-51) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:42:21,965 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-51) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:42:21,965 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-51) spm vds is non responsive, stopping spm selection. >2014-01-09 16:42:25,886 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:42:30,896 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:42:31,963 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-65) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:42:31,965 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-65) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:42:31,968 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-65) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:42:31,969 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-65) spm vds is non responsive, stopping spm selection. >2014-01-09 16:42:31,979 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-64) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:42:31,981 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-64) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:42:31,983 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-64) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:42:31,983 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-64) spm vds is non responsive, stopping spm selection. >2014-01-09 16:42:35,908 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:42:40,920 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:42:41,982 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-77) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:42:41,983 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-77) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:42:41,986 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-77) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:42:41,986 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-77) spm vds is non responsive, stopping spm selection. >2014-01-09 16:42:41,997 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:42:41,999 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:42:42,001 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:42:42,001 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) spm vds is non responsive, stopping spm selection. >2014-01-09 16:42:45,931 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-81) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:42:50,943 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:42:52,000 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-90) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:42:52,001 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-90) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:42:52,003 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-90) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:42:52,004 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-90) spm vds is non responsive, stopping spm selection. >2014-01-09 16:42:52,015 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-89) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:42:52,017 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-89) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:42:52,019 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-89) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:42:52,019 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-89) spm vds is non responsive, stopping spm selection. >2014-01-09 16:42:55,955 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:43:00,967 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:43:02,019 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-1) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:43:02,021 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-1) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:43:02,023 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-1) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:43:02,024 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-1) spm vds is non responsive, stopping spm selection. >2014-01-09 16:43:02,034 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:43:02,036 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:43:02,039 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:43:02,039 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) spm vds is non responsive, stopping spm selection. >2014-01-09 16:43:05,978 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:43:10,991 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:43:12,036 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-9) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:43:12,038 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-9) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:43:12,040 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-9) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:43:12,041 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-9) spm vds is non responsive, stopping spm selection. >2014-01-09 16:43:12,051 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-17) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:43:12,053 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-17) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:43:12,055 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-17) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:43:12,055 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-17) spm vds is non responsive, stopping spm selection. >2014-01-09 16:43:16,003 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:43:21,015 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:43:22,054 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:43:22,055 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:43:22,058 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:43:22,058 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) spm vds is non responsive, stopping spm selection. >2014-01-09 16:43:22,068 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:43:22,069 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:43:22,071 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:43:22,072 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) spm vds is non responsive, stopping spm selection. >2014-01-09 16:43:26,027 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:43:31,040 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:43:32,072 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-35) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:43:32,074 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-35) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:43:32,076 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-35) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:43:32,077 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-35) spm vds is non responsive, stopping spm selection. >2014-01-09 16:43:32,088 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:43:32,090 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:43:32,092 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:43:32,093 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) spm vds is non responsive, stopping spm selection. >2014-01-09 16:43:36,052 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:43:41,064 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:43:42,090 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-56) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:43:42,092 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-56) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:43:42,094 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-56) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:43:42,094 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-56) spm vds is non responsive, stopping spm selection. >2014-01-09 16:43:42,105 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-40) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:43:42,106 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-40) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:43:42,108 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-40) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:43:42,109 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-40) spm vds is non responsive, stopping spm selection. >2014-01-09 16:43:46,076 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:43:51,088 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:43:52,108 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-57) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:43:52,110 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-57) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:43:52,112 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-57) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:43:52,113 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-57) spm vds is non responsive, stopping spm selection. >2014-01-09 16:43:52,123 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:43:52,125 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:43:52,127 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:43:52,127 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) spm vds is non responsive, stopping spm selection. >2014-01-09 16:43:56,100 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:44:01,111 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:44:02,126 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-77) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:44:02,128 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-77) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:44:02,130 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-77) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:44:02,131 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-77) spm vds is non responsive, stopping spm selection. >2014-01-09 16:44:02,142 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:44:02,144 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:44:02,146 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:44:02,147 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-72) spm vds is non responsive, stopping spm selection. >2014-01-09 16:44:06,123 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:44:11,135 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:44:12,144 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:44:12,146 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:44:12,148 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:44:12,149 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) spm vds is non responsive, stopping spm selection. >2014-01-09 16:44:12,159 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-90) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:44:12,161 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-90) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:44:12,163 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-90) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:44:12,163 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-90) spm vds is non responsive, stopping spm selection. >2014-01-09 16:44:16,147 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:44:21,158 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:44:22,164 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-3) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:44:22,166 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-3) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:44:22,168 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-3) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:44:22,169 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-3) spm vds is non responsive, stopping spm selection. >2014-01-09 16:44:22,180 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:44:22,182 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:44:22,184 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:44:22,184 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) spm vds is non responsive, stopping spm selection. >2014-01-09 16:44:26,170 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:44:31,179 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:44:32,184 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-10) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:44:32,186 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-10) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:44:32,188 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-10) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:44:32,189 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-10) spm vds is non responsive, stopping spm selection. >2014-01-09 16:44:32,201 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-23) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:44:32,203 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-23) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:44:32,205 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-23) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:44:32,206 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-23) spm vds is non responsive, stopping spm selection. >2014-01-09 16:44:36,191 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:44:41,201 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:44:42,203 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-28) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:44:42,204 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-28) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:44:42,206 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-28) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:44:42,207 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-28) spm vds is non responsive, stopping spm selection. >2014-01-09 16:44:42,217 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-29) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:44:42,219 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-29) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:44:42,221 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-29) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:44:42,221 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-29) spm vds is non responsive, stopping spm selection. >2014-01-09 16:44:46,214 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:44:51,226 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:44:52,222 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-36) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:44:52,223 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-36) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:44:52,225 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-36) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:44:52,226 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-36) spm vds is non responsive, stopping spm selection. >2014-01-09 16:44:52,237 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-43) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:44:52,239 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-43) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:44:52,241 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-43) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:44:52,241 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-43) spm vds is non responsive, stopping spm selection. >2014-01-09 16:44:56,238 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:45:01,250 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:45:02,240 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-40) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:45:02,242 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-40) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:45:02,245 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-40) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:45:02,246 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-40) spm vds is non responsive, stopping spm selection. >2014-01-09 16:45:02,258 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-48) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:45:02,259 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-48) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:45:02,262 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-48) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:45:02,263 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-48) spm vds is non responsive, stopping spm selection. >2014-01-09 16:45:06,262 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:45:11,274 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:45:12,261 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:45:12,263 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:45:12,265 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:45:12,266 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) spm vds is non responsive, stopping spm selection. >2014-01-09 16:45:12,277 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-60) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:45:12,279 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-60) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:45:12,281 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-60) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:45:12,281 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-60) spm vds is non responsive, stopping spm selection. >2014-01-09 16:45:16,286 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:45:21,299 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:45:22,281 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-69) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:45:22,282 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-69) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:45:22,285 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-69) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:45:22,285 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-69) spm vds is non responsive, stopping spm selection. >2014-01-09 16:45:22,296 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-76) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:45:22,297 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-76) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:45:22,299 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-76) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:45:22,300 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-76) spm vds is non responsive, stopping spm selection. >2014-01-09 16:45:26,309 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:45:31,320 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:45:32,303 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-88) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:45:32,305 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-88) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:45:32,307 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-88) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:45:32,308 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-88) spm vds is non responsive, stopping spm selection. >2014-01-09 16:45:32,318 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-91) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:45:32,320 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-91) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:45:32,322 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-91) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:45:32,323 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-91) spm vds is non responsive, stopping spm selection. >2014-01-09 16:45:36,332 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:45:41,344 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:45:42,321 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-99) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:45:42,322 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-99) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:45:42,325 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-99) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:45:42,325 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-99) spm vds is non responsive, stopping spm selection. >2014-01-09 16:45:42,336 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-7) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:45:42,338 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-7) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:45:42,340 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-7) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:45:42,340 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-7) spm vds is non responsive, stopping spm selection. >2014-01-09 16:45:46,354 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:45:51,366 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:45:52,338 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-6) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:45:52,339 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-6) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:45:52,341 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-6) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:45:52,342 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-6) spm vds is non responsive, stopping spm selection. >2014-01-09 16:45:52,353 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:45:52,354 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:45:52,357 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:45:52,357 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) spm vds is non responsive, stopping spm selection. >2014-01-09 16:45:56,378 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:46:01,387 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-33) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:46:02,355 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-29) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:46:02,357 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-29) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:46:02,359 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-29) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:46:02,360 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-29) spm vds is non responsive, stopping spm selection. >2014-01-09 16:46:02,371 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-25) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:46:02,373 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-25) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:46:02,375 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-25) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:46:02,376 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-25) spm vds is non responsive, stopping spm selection. >2014-01-09 16:46:06,399 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:46:11,411 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:46:12,374 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-43) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:46:12,375 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-43) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:46:12,378 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-43) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:46:12,379 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-43) spm vds is non responsive, stopping spm selection. >2014-01-09 16:46:12,388 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-38) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:46:12,390 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-38) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:46:12,392 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-38) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:46:12,393 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-38) spm vds is non responsive, stopping spm selection. >2014-01-09 16:46:16,423 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:46:21,435 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:46:22,391 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:46:22,393 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:46:22,395 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:46:22,396 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) spm vds is non responsive, stopping spm selection. >2014-01-09 16:46:22,405 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-13) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:46:22,407 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-13) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:46:22,409 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-13) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:46:22,409 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-13) spm vds is non responsive, stopping spm selection. >2014-01-09 16:46:26,447 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:46:27,268 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp--127.0.0.1-8702-9) Running command: LogoutUserCommand internal: false. >2014-01-09 16:46:27,272 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-9) Correlation ID: 5f31d61a, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2014-01-09 16:46:28,610 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-57) Attempting to update VMs/Templates Ovf. >2014-01-09 16:46:28,645 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-66) Start refreshing all users data >2014-01-09 16:46:31,460 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:46:32,408 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:46:32,410 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:46:32,412 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:46:32,413 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) spm vds is non responsive, stopping spm selection. >2014-01-09 16:46:32,424 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-71) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:46:32,426 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-71) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:46:32,428 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-71) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:46:32,429 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-71) spm vds is non responsive, stopping spm selection. >2014-01-09 16:46:36,473 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:46:41,485 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:46:42,425 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-83) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:46:42,427 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-83) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:46:42,429 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-83) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:46:42,429 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-83) spm vds is non responsive, stopping spm selection. >2014-01-09 16:46:42,440 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-92) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:46:42,442 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-92) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:46:42,444 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-92) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:46:42,445 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-92) spm vds is non responsive, stopping spm selection. >2014-01-09 16:46:46,497 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:46:51,508 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:46:52,441 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-4) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:46:52,443 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-4) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:46:52,445 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-4) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:46:52,446 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-4) spm vds is non responsive, stopping spm selection. >2014-01-09 16:46:52,455 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-96) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:46:52,457 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-96) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:46:52,459 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-96) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:46:52,459 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-96) spm vds is non responsive, stopping spm selection. >2014-01-09 16:46:56,520 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:46:58,689 ERROR [org.ovirt.engine.core.bll.adbroker.LdapSearchExceptionHandler] (DefaultQuartzScheduler_Worker-66) Error in communicating with LDAP server cydc-02.cytechservices.local:389; nested exception is javax.naming.CommunicationException: cydc-02.cytechservices.local:389 [Root exception is java.net.SocketTimeoutException: connect timed out] >2014-01-09 16:46:58,692 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-66) Failed ldap search server LDAP://cydc-02.cytechservices.local:389 using user ryan-a@CYTECHSERVICES.LOCAL due to connection timeout. We should try the next server >2014-01-09 16:47:01,532 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:47:02,458 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-16) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:47:02,460 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-16) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:47:02,462 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-16) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:47:02,463 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-16) spm vds is non responsive, stopping spm selection. >2014-01-09 16:47:02,474 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:47:02,476 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:47:02,478 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:47:02,479 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) spm vds is non responsive, stopping spm selection. >2014-01-09 16:47:06,543 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:47:11,556 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:47:12,476 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-23) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:47:12,478 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-23) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:47:12,480 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-23) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:47:12,480 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-23) spm vds is non responsive, stopping spm selection. >2014-01-09 16:47:12,492 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-14) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:47:12,493 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-14) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:47:12,495 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-14) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:47:12,496 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-14) spm vds is non responsive, stopping spm selection. >2014-01-09 16:47:16,566 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:47:21,577 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:47:22,493 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-30) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:47:22,494 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-30) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:47:22,496 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-30) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:47:22,497 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-30) spm vds is non responsive, stopping spm selection. >2014-01-09 16:47:22,508 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:47:22,510 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:47:22,512 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:47:22,512 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) spm vds is non responsive, stopping spm selection. >2014-01-09 16:47:26,588 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:47:31,598 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:47:32,509 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:47:32,510 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:47:32,513 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:47:32,513 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) spm vds is non responsive, stopping spm selection. >2014-01-09 16:47:32,523 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:47:32,524 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:47:32,526 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:47:32,527 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) spm vds is non responsive, stopping spm selection. >2014-01-09 16:47:36,609 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:47:41,622 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:47:42,526 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:47:42,528 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:47:42,530 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:47:42,531 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) spm vds is non responsive, stopping spm selection. >2014-01-09 16:47:42,542 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:47:42,544 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:47:42,546 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:47:42,546 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) spm vds is non responsive, stopping spm selection. >2014-01-09 16:47:46,633 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:47:51,645 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:47:52,545 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-80) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:47:52,547 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-80) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:47:52,549 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-80) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:47:52,550 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-80) spm vds is non responsive, stopping spm selection. >2014-01-09 16:47:52,561 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:47:52,563 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:47:52,565 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:47:52,565 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) spm vds is non responsive, stopping spm selection. >2014-01-09 16:47:54,656 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 16:47:57,665 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 16:48:00,675 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 16:48:02,563 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-85) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:48:02,564 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-85) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:48:02,567 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-85) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:48:02,567 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-85) spm vds is non responsive, stopping spm selection. >2014-01-09 16:48:02,579 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-89) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:48:02,581 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-89) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:48:02,583 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-89) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:48:02,584 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-89) spm vds is non responsive, stopping spm selection. >2014-01-09 16:48:03,685 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 16:48:06,695 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 16:48:09,705 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 16:48:12,581 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-98) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:48:12,583 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-98) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:48:12,585 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-98) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:48:12,585 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-98) spm vds is non responsive, stopping spm selection. >2014-01-09 16:48:12,595 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:48:12,597 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:48:12,599 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:48:12,599 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-95) spm vds is non responsive, stopping spm selection. >2014-01-09 16:48:12,714 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 16:48:16,123 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-12) START, GetHardwareInfoVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02]), log id: 27a339f >2014-01-09 16:48:16,294 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-12) FINISH, GetHardwareInfoVDSCommand, log id: 27a339f >2014-01-09 16:48:16,311 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host ovirt02. >2014-01-09 16:48:16,319 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) vm DC-02 running in db and not running in vds - add to rerun treatment. vds ovirt02 >2014-01-09 16:48:16,320 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) vm VETS running in db and not running in vds - add to rerun treatment. vds ovirt02 >2014-01-09 16:48:16,320 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) vm External_Email running in db and not running in vds - add to rerun treatment. vds ovirt02 >2014-01-09 16:48:16,321 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-12) Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 16:48:16,325 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-12) Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 16:48:16,405 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-12) START, ConnectStorageServerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = ISCSI, connectionList = [{ id: 623aa506-12d1-4437-8e81-f39b2317f3dd, connection: 192.168.101.101, iqn: iqn.2013-11.local.cytechservices.n16000pro:iscsi.hosts-tgt00.raid0, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 21e008ff >2014-01-09 16:48:19,342 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-12) FINISH, ConnectStorageServerVDSCommand, return: {623aa506-12d1-4437-8e81-f39b2317f3dd=0}, log id: 21e008ff >2014-01-09 16:48:19,343 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-12) Host ovirt02 storage connection was succeeded >2014-01-09 16:48:19,345 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-12) START, ConnectStorageServerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 210997b7-9098-48d6-8e97-9689085aa4b0, connection: 192.168.101.101:/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: AUTO, nfsRetrans: null, nfsTimeo: null };]), log id: 23c61700 >2014-01-09 16:48:19,675 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-12) FINISH, ConnectStorageServerVDSCommand, return: {210997b7-9098-48d6-8e97-9689085aa4b0=0}, log id: 23c61700 >2014-01-09 16:48:19,676 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-12) Host ovirt02 storage connection was succeeded >2014-01-09 16:48:19,678 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-12) START, ConnectStorageServerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 4e5d4030-1bd2-4487-b08f-2135eba3f11f, connection: 192.168.101.101:/vms, iqn: null, vfsType: null, mountOptions: null, nfsVersion: V4, nfsRetrans: null, nfsTimeo: null };]), log id: 3dbc714d >2014-01-09 16:48:19,742 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-12) FINISH, ConnectStorageServerVDSCommand, return: {4e5d4030-1bd2-4487-b08f-2135eba3f11f=0}, log id: 3dbc714d >2014-01-09 16:48:19,743 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-12) Host ovirt02 storage connection was succeeded >2014-01-09 16:48:19,746 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-6-thread-50) START, ConnectStoragePoolVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, vds_spm_id = 2, masterDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, masterVersion = 2), log id: 37e74991 >2014-01-09 16:48:22,597 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:48:22,599 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:48:22,601 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:48:22,602 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) spm vds is non responsive, stopping spm selection. >2014-01-09 16:48:22,612 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-9) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:48:22,614 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-9) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 2 >2014-01-09 16:48:22,616 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-9) SPM selection - vds seems as spm ovirt02 >2014-01-09 16:48:22,616 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-9) spm vds is non responsive, stopping spm selection. >2014-01-09 16:48:27,281 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-6-thread-50) FINISH, ConnectStoragePoolVDSCommand, log id: 37e74991 >2014-01-09 16:48:27,298 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-12) START, SetMOMPolicyParametersVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5), log id: 10e969c1 >2014-01-09 16:48:27,374 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-12) FINISH, SetMOMPolicyParametersVDSCommand, log id: 10e969c1 >2014-01-09 16:48:27,378 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host ovirt02. >2014-01-09 16:48:27,385 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-12) Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 16:48:27,390 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-12) Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 16:48:27,399 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Highly Available VM DC-02 failed. It will be restarted automatically. >2014-01-09 16:48:27,400 INFO [org.ovirt.engine.core.bll.VdsEventListener] (DefaultQuartzScheduler_Worker-12) Highly Available VM went down. Attempting to restart. VM Name: DC-02, VM Id:33f88d50-30eb-45c7-9850-733da17ed7a4 >2014-01-09 16:48:27,431 INFO [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-12) Lock Acquired to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 16:48:27,449 WARN [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-12) CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,VM_CANNOT_RUN_FROM_CD_WITHOUT_ACTIVE_STORAGE_DOMAIN_ISO >2014-01-09 16:48:27,450 INFO [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-12) Lock freed to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 16:48:27,455 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Restart of the Highly Available VM DC-02 failed. >2014-01-09 16:48:27,461 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Highly Available VM VETS failed. It will be restarted automatically. >2014-01-09 16:48:27,461 INFO [org.ovirt.engine.core.bll.VdsEventListener] (DefaultQuartzScheduler_Worker-12) Highly Available VM went down. Attempting to restart. VM Name: VETS, VM Id:60ace091-8a4e-4047-9ac0-c0ea95e3c5fa >2014-01-09 16:48:27,478 INFO [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-12) Lock Acquired to object EngineLock [exclusiveLocks= key: 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa value: VM >, sharedLocks= ] >2014-01-09 16:48:27,494 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (DefaultQuartzScheduler_Worker-12) START, IsVmDuringInitiatingVDSCommand( vmId = 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa), log id: c9ffef8 >2014-01-09 16:48:27,494 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (DefaultQuartzScheduler_Worker-12) FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: c9ffef8 >2014-01-09 16:48:27,507 INFO [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-12) Running command: RunVmCommand internal: true. Entities affected : ID: 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa Type: VM >2014-01-09 16:48:27,542 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (DefaultQuartzScheduler_Worker-12) START, IsoPrefixVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 5d96ff78 >2014-01-09 16:48:27,728 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (DefaultQuartzScheduler_Worker-12) FINISH, IsoPrefixVDSCommand, return: /rhev/data-center/mnt/192.168.101.101:_iso/de2fccd8-63cc-4662-8878-077b4cfb810e/images/11111111-1111-1111-1111-111111111111, log id: 5d96ff78 >2014-01-09 16:48:27,742 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (DefaultQuartzScheduler_Worker-12) START, CreateVmVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, vm=VM [VETS]), log id: 19105ee3 >2014-01-09 16:48:27,754 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (DefaultQuartzScheduler_Worker-12) START, CreateVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, vm=VM [VETS]), log id: e24f35c >2014-01-09 16:48:27,851 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (DefaultQuartzScheduler_Worker-12) org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand spiceSslCipherSuite=DEFAULT,memSize=384,kvmEnable=true,smp=1,vmType=kvm,emulatedMachine=rhel6.4.0,keyboardLayout=en-us,memGuaranteedSize=384,pitReinjection=false,nice=0,display=qxl,smartcardEnable=false,smpCoresPerSocket=1,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,timeOffset=0,transparentHugePages=true,vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa,devices=[Ljava.util.HashMap;@28d9b522,acpiEnable=true,vmName=VETS,cpuType=Nehalem,custom={device_d8c45aaf-8d42-45bb-b789-ad4a976ce6bcdevice_4f0ff935-5436-4038-9b3e-7fc1435029e4=VmDevice {vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, deviceId=4f0ff935-5436-4038-9b3e-7fc1435029e4, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}}, device_d8c45aaf-8d42-45bb-b789-ad4a976ce6bcdevice_4f0ff935-5436-4038-9b3e-7fc1435029e4device_0ff45d2d-d30a-43fe-bae0-fc08f6c3ed45device_5c5bbc1b-c354-4c0f-ac5d-055cb60f60b9=VmDevice {vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, deviceId=5c5bbc1b-c354-4c0f-ac5d-055cb60f60b9, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}}, device_d8c45aaf-8d42-45bb-b789-ad4a976ce6bcdevice_4f0ff935-5436-4038-9b3e-7fc1435029e4device_0ff45d2d-d30a-43fe-bae0-fc08f6c3ed45device_5c5bbc1b-c354-4c0f-ac5d-055cb60f60b9device_6047a24a-3e01-4d89-bf39-f6ba4b23dcc9=VmDevice {vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, deviceId=6047a24a-3e01-4d89-bf39-f6ba4b23dcc9, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x04, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}}, device_d8c45aaf-8d42-45bb-b789-ad4a976ce6bc=VmDevice {vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, deviceId=d8c45aaf-8d42-45bb-b789-ad4a976ce6bc, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}}, device_d8c45aaf-8d42-45bb-b789-ad4a976ce6bcdevice_4f0ff935-5436-4038-9b3e-7fc1435029e4device_0ff45d2d-d30a-43fe-bae0-fc08f6c3ed45=VmDevice {vmId=60ace091-8a4e-4047-9ac0-c0ea95e3c5fa, deviceId=0ff45d2d-d30a-43fe-bae0-fc08f6c3ed45, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}}} >2014-01-09 16:48:27,859 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (DefaultQuartzScheduler_Worker-12) FINISH, CreateVDSCommand, log id: e24f35c >2014-01-09 16:48:27,867 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (DefaultQuartzScheduler_Worker-12) FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 19105ee3 >2014-01-09 16:48:27,867 INFO [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-12) Lock freed to object EngineLock [exclusiveLocks= key: 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa value: VM >, sharedLocks= ] >2014-01-09 16:48:27,873 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-12) Correlation ID: 71bdd7be, Job ID: 28f05aa7-dc26-480e-a71a-10a3bebb0c84, Call Stack: null, Custom Event ID: -1, Message: VM VETS was restarted on Host ovirt02 >2014-01-09 16:48:27,880 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Highly Available VM External_Email failed. It will be restarted automatically. >2014-01-09 16:48:27,880 INFO [org.ovirt.engine.core.bll.VdsEventListener] (DefaultQuartzScheduler_Worker-12) Highly Available VM went down. Attempting to restart. VM Name: External_Email, VM Id:872c3dea-6984-48fa-99c3-354fd367c3f9 >2014-01-09 16:48:27,898 INFO [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-12) Lock Acquired to object EngineLock [exclusiveLocks= key: 872c3dea-6984-48fa-99c3-354fd367c3f9 value: VM >, sharedLocks= ] >2014-01-09 16:48:27,915 WARN [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-12) CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,VM_CANNOT_RUN_FROM_CD_WITHOUT_ACTIVE_STORAGE_DOMAIN_ISO >2014-01-09 16:48:27,916 INFO [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-12) Lock freed to object EngineLock [exclusiveLocks= key: 872c3dea-6984-48fa-99c3-354fd367c3f9 value: VM >, sharedLocks= ] >2014-01-09 16:48:27,921 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Restart of the Highly Available VM External_Email failed. >2014-01-09 16:48:30,967 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-15) VM VETS 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa moved from WaitForLaunch --> PoweringUp >2014-01-09 16:48:30,968 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-15) START, FullListVdsCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02], vmIds=[60ace091-8a4e-4047-9ac0-c0ea95e3c5fa]), log id: 7edc58f7 >2014-01-09 16:48:30,978 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-15) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@38c19475, log id: 7edc58f7 >2014-01-09 16:48:30,980 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-15) Received a memballoon Device without an address when processing VM 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa devices, skipping device: {alias=balloon0, specParams={model=none}, device=memballoon, type=balloon} >2014-01-09 16:48:32,615 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) hostFromVds::selectedVds - ovirt01, spmStatus Free, storage pool CSEnt >2014-01-09 16:48:32,615 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) Found spm host bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, host name: ovirt02, according to spmId: 2. >2014-01-09 16:48:32,821 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) SpmStatus on vds bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5: Free >2014-01-09 16:48:32,823 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) starting spm on vds ovirt01, storage pool CSEnt, prevId 2, LVER 14 >2014-01-09 16:48:32,825 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-27) START, SpmStartVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, prevId=2, prevLVER=14, storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=false), log id: 2ac20513 >2014-01-09 16:48:32,839 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-27) spmStart polling started: taskId = bc761e07-a31d-44f8-88fc-4ec27830e257 >2014-01-09 16:48:36,944 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-27) spmStart polling ended: taskId = bc761e07-a31d-44f8-88fc-4ec27830e257 task status = finished >2014-01-09 16:48:37,048 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-27) spmStart polling ended, spm status: SPM >2014-01-09 16:48:37,050 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-27) START, HSMClearTaskVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, taskId=bc761e07-a31d-44f8-88fc-4ec27830e257), log id: 2e9cadc5 >2014-01-09 16:48:37,118 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-27) FINISH, HSMClearTaskVDSCommand, log id: 2e9cadc5 >2014-01-09 16:48:37,119 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-27) FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.businessentities.SpmStatusResult@89cbc1e, log id: 2ac20513 >2014-01-09 16:48:37,122 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) Initialize Irs proxy from vds: 192.168.100.21 >2014-01-09 16:48:37,125 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-27) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host ovirt01 (Address: 192.168.100.21). >2014-01-09 16:48:37,135 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-6-thread-49) START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false), log id: 4e49a669 >2014-01-09 16:48:37,441 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-6-thread-49) -- executeIrsBrokerCommand: Attempting on storage pool 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 16:48:37,443 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (pool-6-thread-49) START, HSMGetAllTasksInfoVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8), log id: 415af23d >2014-01-09 16:48:37,454 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (pool-6-thread-49) FINISH, HSMGetAllTasksInfoVDSCommand, return: [org.ovirt.engine.core.common.asynctasks.AsyncTaskCreationInfo@67d7512d, org.ovirt.engine.core.common.asynctasks.AsyncTaskCreationInfo@6912969d], log id: 415af23d >2014-01-09 16:48:37,455 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-6-thread-49) FINISH, SPMGetAllTasksInfoVDSCommand, return: [org.ovirt.engine.core.common.asynctasks.AsyncTaskCreationInfo@67d7512d, org.ovirt.engine.core.common.asynctasks.AsyncTaskCreationInfo@6912969d], log id: 4e49a669 >2014-01-09 16:48:37,464 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-20) SPMAsyncTask::PollTask: Polling task 3ba786e8-97a9-4d52-915c-7f9a7536eebc (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'cleanSuccess'. >2014-01-09 16:48:37,468 ERROR [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-20) BaseAsyncTask::LogEndTaskFailure: Task 3ba786e8-97a9-4d52-915c-7f9a7536eebc (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended with failure: >-- Result: cleanSuccess >-- Message: VDSGenericException: VDSTaskResultNotSuccessException: TaskState contained successful return code, but a non-success result ('cleanSuccess')., >-- Exception: VDSGenericException: VDSTaskResultNotSuccessException: TaskState contained successful return code, but a non-success result ('cleanSuccess'). >2014-01-09 16:48:37,470 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-20) Task with DB Task ID 84f83f6c-23f1-420e-b5fd-d1e807bcb819 and VDSM Task ID ff40708c-b30c-48a0-a453-29e9447e33d6 is in state Polling. End action for command c472f044-8e48-4fe2-8cfd-d8f81b768ac0 will proceed when all the entitys tasks are completed. >2014-01-09 16:48:37,471 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-20) SPMAsyncTask::PollTask: Polling task ff40708c-b30c-48a0-a453-29e9447e33d6 (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status cleaning. >2014-01-09 16:48:37,473 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (pool-6-thread-49) Discovered 2 tasks on Storage Pool CSEnt, 0 added to manager. >2014-01-09 16:48:47,501 ERROR [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-49) BaseAsyncTask::LogEndTaskFailure: Task 3ba786e8-97a9-4d52-915c-7f9a7536eebc (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended with failure: >-- Result: cleanSuccess >-- Message: VDSGenericException: VDSTaskResultNotSuccessException: TaskState contained successful return code, but a non-success result ('cleanSuccess')., >-- Exception: VDSGenericException: VDSTaskResultNotSuccessException: TaskState contained successful return code, but a non-success result ('cleanSuccess'). >2014-01-09 16:48:47,503 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-49) Task with DB Task ID 84f83f6c-23f1-420e-b5fd-d1e807bcb819 and VDSM Task ID ff40708c-b30c-48a0-a453-29e9447e33d6 is in state Polling. End action for command c472f044-8e48-4fe2-8cfd-d8f81b768ac0 will proceed when all the entitys tasks are completed. >2014-01-09 16:48:47,504 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-49) SPMAsyncTask::PollTask: Polling task ff40708c-b30c-48a0-a453-29e9447e33d6 (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'cleanSuccess'. >2014-01-09 16:48:47,507 ERROR [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-49) BaseAsyncTask::LogEndTaskFailure: Task ff40708c-b30c-48a0-a453-29e9447e33d6 (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended with failure: >-- Result: cleanSuccess >-- Message: VDSGenericException: VDSTaskResultNotSuccessException: TaskState contained successful return code, but a non-success result ('cleanSuccess')., >-- Exception: VDSGenericException: VDSTaskResultNotSuccessException: TaskState contained successful return code, but a non-success result ('cleanSuccess'). >2014-01-09 16:48:47,509 INFO [org.ovirt.engine.core.bll.CommandAsyncTask] (DefaultQuartzScheduler_Worker-49) CommandAsyncTask::EndActionIfNecessary: All tasks of command c472f044-8e48-4fe2-8cfd-d8f81b768ac0 has ended -> executing EndAction >2014-01-09 16:48:47,510 INFO [org.ovirt.engine.core.bll.CommandAsyncTask] (DefaultQuartzScheduler_Worker-49) CommandAsyncTask::EndAction: Ending action for 2 tasks (command ID: c472f044-8e48-4fe2-8cfd-d8f81b768ac0): calling EndAction . >2014-01-09 16:48:47,511 INFO [org.ovirt.engine.core.bll.CommandAsyncTask] (pool-6-thread-49) CommandAsyncTask::EndCommandAction [within thread] context: Attempting to EndAction AddVmFromSnapshot, executionIndex: 0 >2014-01-09 16:48:47,520 ERROR [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (pool-6-thread-49) [2a2843ff] Ending command with failure: org.ovirt.engine.core.bll.AddVmFromSnapshotCommand >2014-01-09 16:48:47,532 ERROR [org.ovirt.engine.core.bll.CopyImageGroupCommand] (pool-6-thread-49) [55104035] Ending command with failure: org.ovirt.engine.core.bll.CopyImageGroupCommand >2014-01-09 16:48:47,541 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (pool-6-thread-49) Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-01-09 16:48:47,554 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (pool-6-thread-49) START, DeleteImageGroupVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, storageDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, imageGroupId = 4cd5c453-12b4-42f4-9a95-a3743e5afbd6, postZeros = false, forceDelete = false), log id: 49e1a757 >2014-01-09 16:48:48,133 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (pool-6-thread-49) Command DeleteImageGroupVDS execution failed. Exception: IrsOperationFailedNoFailoverException: IRSGenericException: IRSErrorException: Image does not exist in domain: 'image=4cd5c453-12b4-42f4-9a95-a3743e5afbd6, domain=cbdefcb2-08ca-400f-84e0-1a6c7d28d763' >2014-01-09 16:48:48,134 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (pool-6-thread-49) FINISH, DeleteImageGroupVDSCommand, log id: 49e1a757 >2014-01-09 16:48:48,135 WARN [org.ovirt.engine.core.bll.RemoveImageCommand] (pool-6-thread-49) The image group with id 4cd5c453-12b4-42f4-9a95-a3743e5afbd6 wasnt actually deleted from the storage domain cbdefcb2-08ca-400f-84e0-1a6c7d28d763 because it didnt exist in it >2014-01-09 16:48:48,188 ERROR [org.ovirt.engine.core.bll.CopyImageGroupCommand] (pool-6-thread-49) [2cafed70] Ending command with failure: org.ovirt.engine.core.bll.CopyImageGroupCommand >2014-01-09 16:48:48,190 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (pool-6-thread-49) Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-01-09 16:48:48,195 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (pool-6-thread-49) START, DeleteImageGroupVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, storageDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, imageGroupId = 51fdfe7d-97eb-4c6b-affa-24eb2b9aa9d4, postZeros = false, forceDelete = false), log id: 12006e59 >2014-01-09 16:48:48,766 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (pool-6-thread-49) Command DeleteImageGroupVDS execution failed. Exception: IrsOperationFailedNoFailoverException: IRSGenericException: IRSErrorException: Image does not exist in domain: 'image=51fdfe7d-97eb-4c6b-affa-24eb2b9aa9d4, domain=cbdefcb2-08ca-400f-84e0-1a6c7d28d763' >2014-01-09 16:48:48,767 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (pool-6-thread-49) FINISH, DeleteImageGroupVDSCommand, log id: 12006e59 >2014-01-09 16:48:48,768 WARN [org.ovirt.engine.core.bll.RemoveImageCommand] (pool-6-thread-49) The image group with id 51fdfe7d-97eb-4c6b-affa-24eb2b9aa9d4 wasnt actually deleted from the storage domain cbdefcb2-08ca-400f-84e0-1a6c7d28d763 because it didnt exist in it >2014-01-09 16:48:48,824 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-49) Correlation ID: 2a2843ff, Job ID: f576e816-56fc-4dca-a9f4-0a283687fb36, Call Stack: null, Custom Event ID: -1, Message: Failed to complete VM syslog00-0 creation. >2014-01-09 16:48:48,825 INFO [org.ovirt.engine.core.bll.CommandAsyncTask] (pool-6-thread-49) CommandAsyncTask::HandleEndActionResult [within thread]: EndAction for action type AddVmFromSnapshot completed, handling the result. >2014-01-09 16:48:48,826 INFO [org.ovirt.engine.core.bll.CommandAsyncTask] (pool-6-thread-49) CommandAsyncTask::HandleEndActionResult [within thread]: EndAction for action type AddVmFromSnapshot succeeded, clearing tasks. >2014-01-09 16:48:48,828 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (pool-6-thread-49) SPMAsyncTask::ClearAsyncTask: Attempting to clear task ff40708c-b30c-48a0-a453-29e9447e33d6 >2014-01-09 16:48:48,829 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (pool-6-thread-49) START, SPMClearTaskVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, taskId = ff40708c-b30c-48a0-a453-29e9447e33d6), log id: 7c7ad13c >2014-01-09 16:48:48,830 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (pool-6-thread-49) START, HSMClearTaskVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, taskId=ff40708c-b30c-48a0-a453-29e9447e33d6), log id: 633ef9fc >2014-01-09 16:48:48,841 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (pool-6-thread-49) FINISH, HSMClearTaskVDSCommand, log id: 633ef9fc >2014-01-09 16:48:48,841 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (pool-6-thread-49) FINISH, SPMClearTaskVDSCommand, log id: 7c7ad13c >2014-01-09 16:48:48,844 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (pool-6-thread-49) BaseAsyncTask::RemoveTaskFromDB: Removed task ff40708c-b30c-48a0-a453-29e9447e33d6 from DataBase >2014-01-09 16:48:48,845 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (pool-6-thread-49) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 3ba786e8-97a9-4d52-915c-7f9a7536eebc >2014-01-09 16:48:48,845 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (pool-6-thread-49) START, SPMClearTaskVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, taskId = 3ba786e8-97a9-4d52-915c-7f9a7536eebc), log id: 3911ca68 >2014-01-09 16:48:48,847 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (pool-6-thread-49) START, HSMClearTaskVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, taskId=3ba786e8-97a9-4d52-915c-7f9a7536eebc), log id: 3eff8f1d >2014-01-09 16:48:48,856 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (pool-6-thread-49) FINISH, HSMClearTaskVDSCommand, log id: 3eff8f1d >2014-01-09 16:48:48,857 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (pool-6-thread-49) FINISH, SPMClearTaskVDSCommand, log id: 3911ca68 >2014-01-09 16:48:48,858 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (pool-6-thread-49) BaseAsyncTask::RemoveTaskFromDB: Removed task 3ba786e8-97a9-4d52-915c-7f9a7536eebc from DataBase >2014-01-09 16:48:48,859 INFO [org.ovirt.engine.core.bll.CommandAsyncTask] (pool-6-thread-49) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity c472f044-8e48-4fe2-8cfd-d8f81b768ac0 >2014-01-09 16:49:03,012 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (ajp--127.0.0.1-8702-6) [2fb389e4] Lock Acquired to object EngineLock [exclusiveLocks= key: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 value: VDS_FENCE >, sharedLocks= ] >2014-01-09 16:49:03,013 WARN [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (ajp--127.0.0.1-8702-6) [2fb389e4] CanDoAction of action FenceVdsManualy failed. Reasons:VAR__TYPE__HOST,VAR__ACTION__MANUAL_FENCE,ACTION_TYPE_FAILED_VDS_NOT_MATCH_VALID_STATUS >2014-01-09 16:49:03,015 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (ajp--127.0.0.1-8702-6) [2fb389e4] Lock freed to object EngineLock [exclusiveLocks= key: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 value: VDS_FENCE >, sharedLocks= ] >2014-01-09 16:49:23,910 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-10) [7445a253] Lock Acquired to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 16:49:23,923 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-10) [7445a253] START, IsVmDuringInitiatingVDSCommand( vmId = 33f88d50-30eb-45c7-9850-733da17ed7a4), log id: 6efc3f71 >2014-01-09 16:49:23,924 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-10) [7445a253] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 6efc3f71 >2014-01-09 16:49:23,948 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-6-thread-49) [7445a253] Running command: RunVmCommand internal: false. Entities affected : ID: 33f88d50-30eb-45c7-9850-733da17ed7a4 Type: VM >2014-01-09 16:49:23,968 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-6-thread-49) [7445a253] START, IsoPrefixVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 35a52361 >2014-01-09 16:49:23,968 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-6-thread-49) [7445a253] FINISH, IsoPrefixVDSCommand, return: /rhev/data-center/mnt/192.168.101.101:_iso/de2fccd8-63cc-4662-8878-077b4cfb810e/images/11111111-1111-1111-1111-111111111111, log id: 35a52361 >2014-01-09 16:49:23,971 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-6-thread-49) [7445a253] START, CreateVmVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, vm=VM [DC-02]), log id: 239cee1a >2014-01-09 16:49:23,982 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-49) [7445a253] START, CreateVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, vm=VM [DC-02]), log id: 69dd8d71 >2014-01-09 16:49:24,043 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-49) [7445a253] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand spiceSslCipherSuite=DEFAULT,memSize=2048,kvmEnable=true,smp=2,vmType=kvm,emulatedMachine=rhel6.4.0,keyboardLayout=en-us,memGuaranteedSize=2048,nice=0,display=qxl,smartcardEnable=false,smpCoresPerSocket=1,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,timeOffset=-18003,transparentHugePages=true,vmId=33f88d50-30eb-45c7-9850-733da17ed7a4,devices=[Ljava.util.HashMap;@5003bb35,acpiEnable=true,vmName=DC-02,cpuType=Nehalem,custom={device_c6495d6f-d8a2-4edd-89ff-6665380c9b90device_cca91e4a-31a7-4137-b064-323fc16eba90device_72b1b147-6d67-4276-846d-615ea7f63e85device_2eb121e2-9dfa-436a-bea3-6a1cf73c52aedevice_23cfefbb-38d5-40d2-83a8-95995529a431=VmDevice {vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, deviceId=23cfefbb-38d5-40d2-83a8-95995529a431, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x04, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}}, device_c6495d6f-d8a2-4edd-89ff-6665380c9b90=VmDevice {vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, deviceId=c6495d6f-d8a2-4edd-89ff-6665380c9b90, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}}, device_c6495d6f-d8a2-4edd-89ff-6665380c9b90device_cca91e4a-31a7-4137-b064-323fc16eba90=VmDevice {vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, deviceId=cca91e4a-31a7-4137-b064-323fc16eba90, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}}, device_c6495d6f-d8a2-4edd-89ff-6665380c9b90device_cca91e4a-31a7-4137-b064-323fc16eba90device_72b1b147-6d67-4276-846d-615ea7f63e85=VmDevice {vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, deviceId=72b1b147-6d67-4276-846d-615ea7f63e85, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}}, device_c6495d6f-d8a2-4edd-89ff-6665380c9b90device_cca91e4a-31a7-4137-b064-323fc16eba90device_72b1b147-6d67-4276-846d-615ea7f63e85device_2eb121e2-9dfa-436a-bea3-6a1cf73c52ae=VmDevice {vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, deviceId=2eb121e2-9dfa-436a-bea3-6a1cf73c52ae, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}}} >2014-01-09 16:49:24,052 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-49) [7445a253] FINISH, CreateVDSCommand, log id: 69dd8d71 >2014-01-09 16:49:24,056 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-6-thread-49) [7445a253] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 239cee1a >2014-01-09 16:49:24,057 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-6-thread-49) [7445a253] Lock freed to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 16:49:24,061 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-49) [7445a253] Correlation ID: 7445a253, Job ID: 97e40e89-d37a-4b4a-a6bb-aa7f94246ea1, Call Stack: null, Custom Event ID: -1, Message: VM DC-02 was started by ryan-a (Host: ovirt01). >2014-01-09 16:49:26,849 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-11) [52f4ba6d] Lock Acquired to object EngineLock [exclusiveLocks= key: 872c3dea-6984-48fa-99c3-354fd367c3f9 value: VM >, sharedLocks= ] >2014-01-09 16:49:26,865 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-11) [52f4ba6d] START, IsVmDuringInitiatingVDSCommand( vmId = 872c3dea-6984-48fa-99c3-354fd367c3f9), log id: 177bac49 >2014-01-09 16:49:26,867 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-11) [52f4ba6d] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 177bac49 >2014-01-09 16:49:26,882 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-6-thread-49) [52f4ba6d] Running command: RunVmCommand internal: false. Entities affected : ID: 872c3dea-6984-48fa-99c3-354fd367c3f9 Type: VM >2014-01-09 16:49:26,899 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-6-thread-49) [52f4ba6d] START, IsoPrefixVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 1de93bfd >2014-01-09 16:49:26,899 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-6-thread-49) [52f4ba6d] FINISH, IsoPrefixVDSCommand, return: /rhev/data-center/mnt/192.168.101.101:_iso/de2fccd8-63cc-4662-8878-077b4cfb810e/images/11111111-1111-1111-1111-111111111111, log id: 1de93bfd >2014-01-09 16:49:26,905 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-6-thread-49) [52f4ba6d] START, CreateVmVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, vm=VM [External_Email]), log id: 576c3c82 >2014-01-09 16:49:26,908 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-49) [52f4ba6d] START, CreateVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, vm=VM [External_Email]), log id: 7f7ee0f8 >2014-01-09 16:49:26,986 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-49) [52f4ba6d] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand spiceSslCipherSuite=DEFAULT,memSize=2048,kvmEnable=true,smp=2,vmType=kvm,emulatedMachine=rhel6.4.0,keyboardLayout=en-us,memGuaranteedSize=2048,pitReinjection=false,nice=0,display=qxl,smartcardEnable=false,smpCoresPerSocket=1,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,timeOffset=-2,transparentHugePages=true,vmId=872c3dea-6984-48fa-99c3-354fd367c3f9,devices=[Ljava.util.HashMap;@4e1c5e25,acpiEnable=true,vmName=External_Email,cpuType=Nehalem,custom={device_5ce857ef-9177-4a3e-971b-8ffa9ec95f83=VmDevice {vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, deviceId=5ce857ef-9177-4a3e-971b-8ffa9ec95f83, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}}, device_5ce857ef-9177-4a3e-971b-8ffa9ec95f83device_3a780492-2cde-4ba3-b32d-485c67293658=VmDevice {vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, deviceId=3a780492-2cde-4ba3-b32d-485c67293658, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}}, device_5ce857ef-9177-4a3e-971b-8ffa9ec95f83device_3a780492-2cde-4ba3-b32d-485c67293658device_0c122810-31aa-41e3-9eaa-8bb7811ae1c9device_fcc988a0-4def-4cf5-9cf4-c26b14b772c8=VmDevice {vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, deviceId=fcc988a0-4def-4cf5-9cf4-c26b14b772c8, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}}, device_5ce857ef-9177-4a3e-971b-8ffa9ec95f83device_3a780492-2cde-4ba3-b32d-485c67293658device_0c122810-31aa-41e3-9eaa-8bb7811ae1c9=VmDevice {vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, deviceId=0c122810-31aa-41e3-9eaa-8bb7811ae1c9, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}}, device_5ce857ef-9177-4a3e-971b-8ffa9ec95f83device_3a780492-2cde-4ba3-b32d-485c67293658device_0c122810-31aa-41e3-9eaa-8bb7811ae1c9device_fcc988a0-4def-4cf5-9cf4-c26b14b772c8device_9c7b35ca-aa51-4928-b3ba-52a3b188c041=VmDevice {vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, deviceId=9c7b35ca-aa51-4928-b3ba-52a3b188c041, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x04, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}}} >2014-01-09 16:49:26,995 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-49) [52f4ba6d] FINISH, CreateVDSCommand, log id: 7f7ee0f8 >2014-01-09 16:49:26,999 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-6-thread-49) [52f4ba6d] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 576c3c82 >2014-01-09 16:49:27,000 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-6-thread-49) [52f4ba6d] Lock freed to object EngineLock [exclusiveLocks= key: 872c3dea-6984-48fa-99c3-354fd367c3f9 value: VM >, sharedLocks= ] >2014-01-09 16:49:27,004 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-49) [52f4ba6d] Correlation ID: 52f4ba6d, Job ID: f1e83eae-b7e7-4e03-b303-5a8fe0060bf1, Call Stack: null, Custom Event ID: -1, Message: VM External_Email was started by ryan-a (Host: ovirt01). >2014-01-09 16:49:28,021 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) VM DC-02 33f88d50-30eb-45c7-9850-733da17ed7a4 moved from WaitForLaunch --> PoweringUp >2014-01-09 16:49:28,047 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-96) START, FullListVdsCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01], vmIds=[33f88d50-30eb-45c7-9850-733da17ed7a4, 872c3dea-6984-48fa-99c3-354fd367c3f9]), log id: 5c4771d2 >2014-01-09 16:49:28,060 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-96) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@60bb1adc, log id: 5c4771d2 >2014-01-09 16:49:28,063 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) Received a memballoon Device without an address when processing VM 872c3dea-6984-48fa-99c3-354fd367c3f9 devices, skipping device: {specParams={model=virtio}, target=2097152, device=memballoon, type=balloon, deviceId=6fe0329d-0ea4-4fb4-bcf8-e36ebebe9aba} >2014-01-09 16:49:28,482 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) VM VETS 60ace091-8a4e-4047-9ac0-c0ea95e3c5fa moved from PoweringUp --> Up >2014-01-09 16:49:28,487 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-93) Correlation ID: 71bdd7be, Job ID: 28f05aa7-dc26-480e-a71a-10a3bebb0c84, Call Stack: null, Custom Event ID: -1, Message: VM VETS was restarted on Host ovirt02 >2014-01-09 16:49:31,114 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) VM External_Email 872c3dea-6984-48fa-99c3-354fd367c3f9 moved from WaitForLaunch --> PoweringUp >2014-01-09 16:49:31,115 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-7) START, FullListVdsCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01], vmIds=[872c3dea-6984-48fa-99c3-354fd367c3f9]), log id: 70eef7e3 >2014-01-09 16:49:31,125 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-7) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@6bbdad36, log id: 70eef7e3 >2014-01-09 16:49:57,295 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (DefaultQuartzScheduler_Worker-36) Setting new tasks map. The map contains now 0 tasks >2014-01-09 16:49:57,296 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (DefaultQuartzScheduler_Worker-36) Cleared all tasks of pool 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d. >2014-01-09 16:50:25,817 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) VM DC-02 33f88d50-30eb-45c7-9850-733da17ed7a4 moved from PoweringUp --> Up >2014-01-09 16:50:25,822 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-77) Correlation ID: 7445a253, Job ID: 97e40e89-d37a-4b4a-a6bb-aa7f94246ea1, Call Stack: null, Custom Event ID: -1, Message: VM DC-02 started on Host ovirt01 >2014-01-09 16:50:28,863 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-83) VM External_Email 872c3dea-6984-48fa-99c3-354fd367c3f9 moved from PoweringUp --> Up >2014-01-09 16:50:28,878 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-83) Correlation ID: 52f4ba6d, Job ID: f1e83eae-b7e7-4e03-b303-5a8fe0060bf1, Call Stack: null, Custom Event ID: -1, Message: VM External_Email started on Host ovirt01 >2014-01-09 16:51:26,713 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetIsoListVDSCommand] (pool-6-thread-50) START, GetIsoListVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false), log id: 4567f25 >2014-01-09 16:51:26,715 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HsmGetIsoListVDSCommand] (pool-6-thread-50) START, HsmGetIsoListVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 72b156ad >2014-01-09 16:51:26,760 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HsmGetIsoListVDSCommand] (pool-6-thread-50) FINISH, HsmGetIsoListVDSCommand, return: [virtio-win-0.1-52.iso, centos64x64netinst.iso, virtio-win-0.1-65.iso, winserve2012.iso, kali-linux-1.0.5-amd641.iso, Server2008R2SP1-en-US-July2013.iso, win7.iso, virt-p2v-0.9.1-2.20130730.1.el6.centos.iso, zentyal-3.2-amd64.iso, Elastix-2.4.0-Stable-x86_64-bin-04feb2013.iso, EXCH201064.ISO], log id: 72b156ad >2014-01-09 16:51:26,762 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetIsoListVDSCommand] (pool-6-thread-50) FINISH, GetIsoListVDSCommand, return: [virtio-win-0.1-52.iso, centos64x64netinst.iso, virtio-win-0.1-65.iso, winserve2012.iso, kali-linux-1.0.5-amd641.iso, Server2008R2SP1-en-US-July2013.iso, win7.iso, virt-p2v-0.9.1-2.20130730.1.el6.centos.iso, zentyal-3.2-amd64.iso, Elastix-2.4.0-Stable-x86_64-bin-04feb2013.iso, EXCH201064.ISO], log id: 4567f25 >2014-01-09 16:51:26,777 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (pool-6-thread-50) Finished automatic refresh process for ISO file type with success, for storage domain id de2fccd8-63cc-4662-8878-077b4cfb810e. >2014-01-09 16:51:44,236 WARN [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ajp--127.0.0.1-8702-6) The message key AddVmFromSnapshot is missing from bundles/ExecutionMessages >2014-01-09 16:51:44,250 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (ajp--127.0.0.1-8702-6) Lock Acquired to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 16:51:44,401 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (ajp--127.0.0.1-8702-6) Running command: AddVmFromSnapshotCommand internal: false. Entities affected : ID: ac664a0e-2058-44ec-a240-be2adb31bd87 Type: VdsGroups, ID: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 Type: VM, ID: cbdefcb2-08ca-400f-84e0-1a6c7d28d763 Type: Storage, ID: cbdefcb2-08ca-400f-84e0-1a6c7d28d763 Type: Storage, ID: ac664a0e-2058-44ec-a240-be2adb31bd87 Type: VdsGroups >2014-01-09 16:51:44,421 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (ajp--127.0.0.1-8702-6) Locking VM(id = 565c5082-b8a3-4946-8ae2-6d3330c8ef9b) with compensation. >2014-01-09 16:51:44,422 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (ajp--127.0.0.1-8702-6) START, SetVmStatusVDSCommand( vmId = 565c5082-b8a3-4946-8ae2-6d3330c8ef9b, status = ImageLocked), log id: 458a4817 >2014-01-09 16:51:44,425 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (ajp--127.0.0.1-8702-6) FINISH, SetVmStatusVDSCommand, log id: 458a4817 >2014-01-09 16:51:44,428 INFO [org.ovirt.engine.core.bll.AddVmFromSnapshotCommand] (ajp--127.0.0.1-8702-6) Lock freed to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 16:51:44,444 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (ajp--127.0.0.1-8702-6) Running command: CopyImageGroupCommand internal: true. Entities affected : ID: cbdefcb2-08ca-400f-84e0-1a6c7d28d763 Type: Storage >2014-01-09 16:51:44,459 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp--127.0.0.1-8702-6) START, CopyImageVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, storageDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, imageGroupId = de474e2f-5669-47ac-9103-692fd87aad30, imageId = 40742666-bb4f-4741-9d1e-08dddb2dba27, dstImageGroupId = 3736f6c1-f225-4598-ba79-66dc9236b101, vmId = 565c5082-b8a3-4946-8ae2-6d3330c8ef9b, dstImageId = 278581b5-a6d7-497b-b526-d1f65220a205, imageDescription = , dstStorageDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, copyVolumeType = LeafVol, volumeFormat = RAW, preallocate = Preallocated, postZero = false, force = false), log id: 63f81547 >2014-01-09 16:51:44,461 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp--127.0.0.1-8702-6) -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID >2014-01-09 16:51:44,461 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp--127.0.0.1-8702-6) -- copyImage parameters: > sdUUID=cbdefcb2-08ca-400f-84e0-1a6c7d28d763 > spUUID=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d > vmGUID=565c5082-b8a3-4946-8ae2-6d3330c8ef9b > srcImageGUID=de474e2f-5669-47ac-9103-692fd87aad30 > srcVolUUID=40742666-bb4f-4741-9d1e-08dddb2dba27 > dstImageGUID=3736f6c1-f225-4598-ba79-66dc9236b101 > dstVolUUID=278581b5-a6d7-497b-b526-d1f65220a205 > descr= > dstSdUUID=cbdefcb2-08ca-400f-84e0-1a6c7d28d763 >2014-01-09 16:51:45,564 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp--127.0.0.1-8702-6) FINISH, CopyImageVDSCommand, return: 00000000-0000-0000-0000-000000000000, log id: 63f81547 >2014-01-09 16:51:45,590 INFO [org.ovirt.engine.core.bll.CommandAsyncTask] (ajp--127.0.0.1-8702-6) CommandAsyncTask::Adding CommandMultiAsyncTasks object for command ab5e019a-1ee3-403d-8434-cac5fcdc5400 >2014-01-09 16:51:45,591 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (ajp--127.0.0.1-8702-6) CommandMultiAsyncTasks::AttachTask: Attaching task 1f5c4ddb-a230-4107-b13d-5a047bca36e4 to command ab5e019a-1ee3-403d-8434-cac5fcdc5400. >2014-01-09 16:51:45,607 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (ajp--127.0.0.1-8702-6) Adding task 1f5c4ddb-a230-4107-b13d-5a047bca36e4 (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-01-09 16:51:45,619 INFO [org.ovirt.engine.core.bll.CopyImageGroupCommand] (ajp--127.0.0.1-8702-6) Running command: CopyImageGroupCommand internal: true. Entities affected : ID: cbdefcb2-08ca-400f-84e0-1a6c7d28d763 Type: Storage >2014-01-09 16:51:45,628 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp--127.0.0.1-8702-6) START, CopyImageVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, storageDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, imageGroupId = d8f95faa-2402-4906-b6fe-d560edf56a4a, imageId = bd6325e5-934e-446d-a53e-1d5113188611, dstImageGroupId = 95b6fcb0-cde9-40e6-a217-6dfbe9693d5f, vmId = 565c5082-b8a3-4946-8ae2-6d3330c8ef9b, dstImageId = a3169e66-dd65-4412-9107-2c4ec8f47ed3, imageDescription = , dstStorageDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, copyVolumeType = LeafVol, volumeFormat = RAW, preallocate = Preallocated, postZero = false, force = false), log id: 4efd5a2e >2014-01-09 16:51:45,630 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp--127.0.0.1-8702-6) -- executeIrsBrokerCommand: calling 'copyImage' with two new parameters: description and UUID >2014-01-09 16:51:45,630 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp--127.0.0.1-8702-6) -- copyImage parameters: > sdUUID=cbdefcb2-08ca-400f-84e0-1a6c7d28d763 > spUUID=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d > vmGUID=565c5082-b8a3-4946-8ae2-6d3330c8ef9b > srcImageGUID=d8f95faa-2402-4906-b6fe-d560edf56a4a > srcVolUUID=bd6325e5-934e-446d-a53e-1d5113188611 > dstImageGUID=95b6fcb0-cde9-40e6-a217-6dfbe9693d5f > dstVolUUID=a3169e66-dd65-4412-9107-2c4ec8f47ed3 > descr= > dstSdUUID=cbdefcb2-08ca-400f-84e0-1a6c7d28d763 >2014-01-09 16:51:46,819 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (ajp--127.0.0.1-8702-6) FINISH, CopyImageVDSCommand, return: 00000000-0000-0000-0000-000000000000, log id: 4efd5a2e >2014-01-09 16:51:46,827 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (ajp--127.0.0.1-8702-6) CommandMultiAsyncTasks::AttachTask: Attaching task 763d34b0-8132-461f-914b-e329ef9a052e to command ab5e019a-1ee3-403d-8434-cac5fcdc5400. >2014-01-09 16:51:46,852 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (ajp--127.0.0.1-8702-6) Adding task 763d34b0-8132-461f-914b-e329ef9a052e (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-01-09 16:51:46,896 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-6) Correlation ID: 38f9c960, Job ID: 5dded8ac-4cff-4d09-93e3-26645f0a9389, Call Stack: null, Custom Event ID: -1, Message: VM syslog00-1 creation was initiated by ryan-a. >2014-01-09 16:51:46,897 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (ajp--127.0.0.1-8702-6) BaseAsyncTask::StartPollingTask: Starting to poll task 1f5c4ddb-a230-4107-b13d-5a047bca36e4. >2014-01-09 16:51:46,898 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (ajp--127.0.0.1-8702-6) BaseAsyncTask::StartPollingTask: Starting to poll task 763d34b0-8132-461f-914b-e329ef9a052e. >2014-01-09 16:51:47,513 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (DefaultQuartzScheduler_Worker-97) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-01-09 16:51:47,642 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-97) SPMAsyncTask::PollTask: Polling task 763d34b0-8132-461f-914b-e329ef9a052e (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-01-09 16:51:47,644 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-97) SPMAsyncTask::PollTask: Polling task 1f5c4ddb-a230-4107-b13d-5a047bca36e4 (Parent Command AddVmFromSnapshot, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-01-09 16:51:47,645 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (DefaultQuartzScheduler_Worker-97) Finished polling Tasks, will poll again in 10 seconds. >2014-01-09 16:55:52,275 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp--127.0.0.1-8702-5) START, GetVmsInfoVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, storageDomainId = 891d27cc-e65c-4954-a7b8-241360a30440, vmIdList = null), log id: 15af676f >2014-01-09 16:55:52,498 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp--127.0.0.1-8702-5) FINISH, GetVmsInfoVDSCommand, log id: 15af676f >2014-01-09 16:55:55,720 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp--127.0.0.1-8702-8) START, GetVmsInfoVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, storageDomainId = 891d27cc-e65c-4954-a7b8-241360a30440, vmIdList = null), log id: 49751471 >2014-01-09 16:55:55,772 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp--127.0.0.1-8702-8) FINISH, GetVmsInfoVDSCommand, log id: 49751471 >2014-01-09 16:55:58,504 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp--127.0.0.1-8702-3) START, GetVmsInfoVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, storageDomainId = 891d27cc-e65c-4954-a7b8-241360a30440, vmIdList = null), log id: 4ef042a >2014-01-09 16:55:58,552 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp--127.0.0.1-8702-3) FINISH, GetVmsInfoVDSCommand, log id: 4ef042a >2014-01-09 16:55:59,556 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp--127.0.0.1-8702-2) START, GetVmsInfoVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, storageDomainId = 891d27cc-e65c-4954-a7b8-241360a30440, vmIdList = null), log id: 738dcf40 >2014-01-09 16:55:59,604 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp--127.0.0.1-8702-2) FINISH, GetVmsInfoVDSCommand, log id: 738dcf40 >2014-01-09 16:58:12,747 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-24) Command SpmStatusVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:58:12,751 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-24) Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 16:58:12,761 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-24) Correlation ID: 7f9d1bf1, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center CSEnt. Setting Data Center status to Non Responsive (On host ovirt01, Error: Network error during communication with the Host.). >2014-01-09 16:58:13,673 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand] (DefaultQuartzScheduler_Worker-27) Command ListVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:58:13,677 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-27) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:58:14,768 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-24) IrsBroker::Failed::GetStoragePoolInfoVDS due to: SocketTimeoutException: connect timed out >2014-01-09 16:58:16,775 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-24) Command SpmStatusVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:58:16,776 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-24) Irs placed on server f2d7a8a9-e77e-4e9a-8031-23a0731407c8 failed. Proceed Failover >2014-01-09 16:58:18,686 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:58:18,689 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-30) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:58:23,699 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:58:23,702 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-34) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:58:28,711 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:58:28,714 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-36) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:58:33,723 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:58:33,726 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-44) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:58:36,810 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-24) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 16:58:36,812 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-24) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 16:58:36,815 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-24) SPM selection - vds seems as spm ovirt01 >2014-01-09 16:58:36,817 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-24) START, SpmStopVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 1bde6d60 >2014-01-09 16:58:36,820 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-24) SpmStopVDSCommand:: vds ovirt01 is in Connecting status - not performing spm stop, pool id 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 16:58:36,822 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-24) FINISH, SpmStopVDSCommand, log id: 1bde6d60 >2014-01-09 16:58:36,823 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-24) spm stop on spm failed, stopping spm selection! >2014-01-09 16:58:38,734 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:58:38,737 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-45) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:58:43,746 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-55) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:58:43,749 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-55) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:58:48,760 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:58:48,763 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-54) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:58:53,772 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:58:53,774 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-40) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:58:56,854 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 16:58:56,857 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 16:58:56,859 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) SPM selection - vds seems as spm ovirt01 >2014-01-09 16:58:56,861 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-21) START, SpmStopVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 568ef80 >2014-01-09 16:58:56,863 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-21) SpmStopVDSCommand:: vds ovirt01 is in Connecting status - not performing spm stop, pool id 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 16:58:56,864 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-21) FINISH, SpmStopVDSCommand, log id: 568ef80 >2014-01-09 16:58:56,864 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) spm stop on spm failed, stopping spm selection! >2014-01-09 16:58:58,783 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:58:58,785 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-64) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:59:03,793 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:59:03,796 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-75) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:59:08,803 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:59:08,806 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-78) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:59:13,817 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:59:13,820 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-76) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:59:16,895 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-52) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 16:59:16,901 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-52) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 16:59:16,904 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-52) SPM selection - vds seems as spm ovirt01 >2014-01-09 16:59:16,906 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-52) START, SpmStopVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 2499337b >2014-01-09 16:59:16,908 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-52) SpmStopVDSCommand:: vds ovirt01 is in Connecting status - not performing spm stop, pool id 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 16:59:16,910 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-52) FINISH, SpmStopVDSCommand, log id: 2499337b >2014-01-09 16:59:16,911 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-52) spm stop on spm failed, stopping spm selection! >2014-01-09 16:59:18,830 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-87) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:59:18,832 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-87) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:59:23,841 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:59:23,844 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-83) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:59:28,854 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:59:28,856 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-85) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 16:59:33,865 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:59:33,867 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) Server failed to respond, vds_id = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds_name = ovirt01, vm_count = 3, spm_status = SPM, non-responsive_timeout (seconds) = 81, error = java.net.SocketTimeoutException: connect timed out >2014-01-09 16:59:33,874 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) Correlation ID: null, Call Stack: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.SocketTimeoutException: connect timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:102) > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:28) > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:536) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:492) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.Refresh(VdsUpdateRunTimeInfo.java:343) > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:237) > at sun.reflect.GeneratedMethodAccessor50.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >Caused by: java.net.SocketTimeoutException: connect timed out > at java.net.PlainSocketImpl.socketConnect(Native Method) > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) > at java.net.Socket.connect(Socket.java:579) > at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:618) > at org.ovirt.engine.core.utils.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:288) > at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) > at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361) > at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) > at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) > at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) > at org.apache.xmlrpc.client.XmlRpcCommonsTransport.writeRequest(XmlRpcCommonsTransport.java:227) > at org.apache.xmlrpc.client.XmlRpcStreamTransport.sendRequest(XmlRpcStreamTransport.java:151) > at org.apache.xmlrpc.client.XmlRpcHttpTransport.sendRequest(XmlRpcHttpTransport.java:143) > at org.apache.xmlrpc.client.XmlRpcClientWorker.execute(XmlRpcClientWorker.java:56) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:167) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:137) > at org.apache.xmlrpc.client.XmlRpcClient.execute(XmlRpcClient.java:126) > at org.apache.xmlrpc.client.util.ClientFactory$1.invoke(ClientFactory.java:140) > at com.sun.proxy.$Proxy52.getVdsCapabilities(Unknown Source) > at sun.reflect.GeneratedMethodAccessor228.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcUtils$AsyncProxy$InternalCallable.call(XmlRpcUtils.java:225) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:744) >, Custom Event ID: -1, Message: Host ovirt01 is non responsive. >2014-01-09 16:59:33,891 INFO [org.ovirt.engine.core.bll.VdsEventListener] (pool-6-thread-48) ResourceManager::vdsNotResponding entered for Host f2d7a8a9-e77e-4e9a-8031-23a0731407c8, 192.168.100.21 >2014-01-09 16:59:33,902 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (pool-6-thread-48) Running command: SshSoftFencingCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 16:59:34,863 ERROR [org.ovirt.engine.core.bll.SshSoftFencingCommand] (pool-6-thread-48) SSH connection to host 192.168.100.21 failed: org.apache.sshd.common.RuntimeSshException: Failed to get the session.: org.apache.sshd.common.RuntimeSshException: Failed to get the session. > at org.apache.sshd.client.future.DefaultConnectFuture.getSession(DefaultConnectFuture.java:43) > at org.ovirt.engine.core.utils.ssh.SSHClient.connect(SSHClient.java:288) [utils.jar:] > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:59) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeSshSoftFencingCommand(SshSoftFencingCommand.java:73) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeCommand(SshSoftFencingCommand.java:43) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1135) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1220) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1896) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1240) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:366) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:416) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:395) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:636) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:1.1.1.Final] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view10.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.VdsEventListener$1.run(VdsEventListener.java:132) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:71) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_45] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_45] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_45] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_45] > at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_45] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_45] > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735) [rt.jar:1.7.0_45] > at org.apache.mina.transport.socket.nio.NioSocketConnector.finishConnect(NioSocketConnector.java:223) [mina-core.jar:] > at org.apache.mina.transport.socket.nio.NioSocketConnector.finishConnect(NioSocketConnector.java:45) [mina-core.jar:] > at org.apache.mina.core.polling.AbstractPollingIoConnector.processConnections(AbstractPollingIoConnector.java:427) [mina-core.jar:] > at org.apache.mina.core.polling.AbstractPollingIoConnector.access$500(AbstractPollingIoConnector.java:62) [mina-core.jar:] > at org.apache.mina.core.polling.AbstractPollingIoConnector$Connector.run(AbstractPollingIoConnector.java:476) [mina-core.jar:] > at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64) [mina-core.jar:] > ... 3 more > >2014-01-09 16:59:34,924 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-48) START, SetVmStatusVDSCommand( vmId = 33f88d50-30eb-45c7-9850-733da17ed7a4, status = Unknown), log id: 6b6763b3 >2014-01-09 16:59:34,927 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-48) FINISH, SetVmStatusVDSCommand, log id: 6b6763b3 >2014-01-09 16:59:34,951 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM DC-02 was set to the Unknown status. >2014-01-09 16:59:34,952 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-48) START, SetVmStatusVDSCommand( vmId = 872c3dea-6984-48fa-99c3-354fd367c3f9, status = Unknown), log id: 2dddda39 >2014-01-09 16:59:34,954 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-48) FINISH, SetVmStatusVDSCommand, log id: 2dddda39 >2014-01-09 16:59:34,959 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM External_Email was set to the Unknown status. >2014-01-09 16:59:34,960 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-48) START, SetVmStatusVDSCommand( vmId = b6972001-2bd6-4cea-9c15-c046674754d4, status = Unknown), log id: 79d74267 >2014-01-09 16:59:34,962 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-48) FINISH, SetVmStatusVDSCommand, log id: 79d74267 >2014-01-09 16:59:34,967 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM DC-01 was set to the Unknown status. >2014-01-09 16:59:34,968 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (pool-6-thread-48) Fence failed on vds ovirt01 which is spm of pool 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d - moving pool to non operational >2014-01-09 16:59:34,970 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (pool-6-thread-48) Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 16:59:34,977 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) Correlation ID: 4e47b570, Call Stack: null, Custom Event ID: -1, Message: Fencing failed on Storage Pool Manager ovirt01 for Data Center CSEnt. Setting status to Non-Operational. >2014-01-09 16:59:34,978 ERROR [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (pool-6-thread-48) Failed to run Fence script on vds:ovirt01, VMs moved to UnKnown instead. >2014-01-09 16:59:34,981 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host ovirt01 became non responsive. It has no power management configured. Please check the host status, manually reboot it, and click "Confirm Host Has Been Rebooted" >2014-01-09 16:59:34,983 WARN [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (pool-6-thread-48) CanDoAction of action VdsNotRespondingTreatment failed. Reasons:VDS_FENCE_DISABLED >2014-01-09 16:59:36,947 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-71) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 16:59:36,950 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-71) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 16:59:36,953 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-71) SPM selection - vds seems as spm ovirt01 >2014-01-09 16:59:36,954 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-71) spm vds is non responsive, stopping spm selection. >2014-01-09 16:59:38,901 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:59:43,917 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:59:46,960 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-2) Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 16:59:46,968 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-2) Correlation ID: 32daaa24, Call Stack: null, Custom Event ID: -1, Message: Data Center is being initialized, please wait for initialization to complete. >2014-01-09 16:59:48,928 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:59:53,938 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 16:59:58,230 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-01-09 17:00:03,240 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:00:06,999 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-2) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:00:07,003 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-2) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:00:07,006 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-2) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:00:07,007 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-2) spm vds is non responsive, stopping spm selection. >2014-01-09 17:00:08,251 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:00:13,262 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-33) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:00:18,273 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:00:23,283 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:00:27,041 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:00:27,043 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:00:27,046 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:00:27,047 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) spm vds is non responsive, stopping spm selection. >2014-01-09 17:00:28,293 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:00:33,304 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:00:38,315 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:00:43,325 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:00:47,078 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-30) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:00:47,080 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-30) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:00:47,083 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-30) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:00:47,084 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-30) spm vds is non responsive, stopping spm selection. >2014-01-09 17:00:48,335 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:00:53,345 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-57) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:00:58,357 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-21) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:01:03,373 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:01:07,118 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:01:07,121 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:01:07,124 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:01:07,125 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) spm vds is non responsive, stopping spm selection. >2014-01-09 17:01:08,385 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:01:13,398 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:01:18,412 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:01:23,425 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:01:27,142 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-64) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:01:27,144 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-64) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:01:27,147 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-64) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:01:27,148 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-64) spm vds is non responsive, stopping spm selection. >2014-01-09 17:01:28,437 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:01:33,450 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:01:38,463 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:01:41,476 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:01:44,488 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:01:47,183 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-81) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:01:47,185 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-81) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:01:47,189 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-81) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:01:47,190 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-81) spm vds is non responsive, stopping spm selection. >2014-01-09 17:01:47,499 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:01:50,509 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:01:53,519 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:01:56,530 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:01:59,540 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:02:02,549 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:02:05,559 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-33) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:02:07,225 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:02:07,227 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:02:07,229 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:02:07,230 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) spm vds is non responsive, stopping spm selection. >2014-01-09 17:02:08,568 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:02:11,580 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:02:14,589 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:02:17,602 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:02:20,611 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:02:23,620 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:02:26,630 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:02:27,258 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-26) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:02:27,260 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-26) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:02:27,262 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-26) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:02:27,263 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-26) spm vds is non responsive, stopping spm selection. >2014-01-09 17:02:29,640 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:02:32,648 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:02:35,657 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-43) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:02:38,668 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:02:41,678 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-41) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:02:44,687 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:02:47,305 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-37) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:02:47,307 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-37) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:02:47,310 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-37) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:02:47,310 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-37) spm vds is non responsive, stopping spm selection. >2014-01-09 17:02:47,697 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:02:50,707 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:02:53,717 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:02:56,727 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:02:59,737 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:03:02,746 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:03:05,755 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:03:07,359 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-30) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:03:07,361 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-30) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:03:07,363 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-30) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:03:07,364 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-30) spm vds is non responsive, stopping spm selection. >2014-01-09 17:03:08,764 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:03:11,775 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:03:14,784 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-88) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:03:17,794 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:03:20,804 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:03:23,813 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:03:26,822 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:03:27,391 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-65) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:03:27,393 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-65) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:03:27,395 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-65) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:03:27,395 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-65) spm vds is non responsive, stopping spm selection. >2014-01-09 17:03:29,866 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:03:32,875 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:03:35,885 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:03:38,895 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:03:41,908 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:03:44,917 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:03:47,422 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-82) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:03:47,424 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-82) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:03:47,426 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-82) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:03:47,427 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-82) spm vds is non responsive, stopping spm selection. >2014-01-09 17:03:47,928 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:03:50,936 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:03:53,946 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:03:56,956 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:03:59,966 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:04:02,976 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:04:05,984 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:04:07,458 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-63) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:04:07,460 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-63) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:04:07,462 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-63) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:04:07,463 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-63) spm vds is non responsive, stopping spm selection. >2014-01-09 17:04:08,993 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:04:12,003 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:04:15,012 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:04:18,023 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-56) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:04:21,031 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:04:24,040 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-43) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:04:27,050 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:04:27,495 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-39) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:04:27,497 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-39) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:04:27,499 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-39) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:04:27,500 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-39) spm vds is non responsive, stopping spm selection. >2014-01-09 17:04:30,058 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:04:33,067 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:04:36,077 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:04:39,087 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:04:42,096 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:04:45,106 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:04:47,527 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-55) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:04:47,530 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-55) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:04:47,532 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-55) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:04:47,533 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-55) spm vds is non responsive, stopping spm selection. >2014-01-09 17:04:48,115 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:04:51,126 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:04:54,136 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:04:57,145 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:05:00,155 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:05:03,165 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:05:06,177 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-89) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:05:07,566 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:05:07,568 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:05:07,570 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:05:07,571 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) spm vds is non responsive, stopping spm selection. >2014-01-09 17:05:09,187 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:05:12,197 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:05:15,207 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:05:18,217 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-81) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:05:21,227 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:05:24,236 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:05:27,246 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:05:27,594 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-88) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:05:27,597 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-88) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:05:27,600 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-88) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:05:27,600 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-88) spm vds is non responsive, stopping spm selection. >2014-01-09 17:05:30,256 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:05:33,267 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:05:36,276 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:05:39,285 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:05:42,295 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:05:45,305 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:05:47,633 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-1) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:05:47,636 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-1) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:05:47,638 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-1) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:05:47,639 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-1) spm vds is non responsive, stopping spm selection. >2014-01-09 17:05:48,314 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:05:51,324 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:05:54,333 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:05:57,343 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-33) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:06:00,353 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:06:03,362 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:06:06,372 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:06:07,664 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:06:07,667 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:06:07,669 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:06:07,669 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-27) spm vds is non responsive, stopping spm selection. >2014-01-09 17:06:09,382 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:06:12,392 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-41) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:06:15,402 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:06:18,412 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:06:21,421 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:06:24,431 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:06:27,442 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-57) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:06:27,702 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:06:27,704 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:06:27,706 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:06:27,707 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) spm vds is non responsive, stopping spm selection. >2014-01-09 17:06:30,454 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:06:33,462 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:06:36,472 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:06:39,482 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:06:42,493 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:06:45,502 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-89) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:06:47,726 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:06:47,728 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:06:47,731 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:06:47,732 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) spm vds is non responsive, stopping spm selection. >2014-01-09 17:06:48,510 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:06:51,519 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:06:54,529 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:06:57,539 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:07:00,549 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:07:03,558 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:07:06,567 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:07:07,760 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-91) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:07:07,762 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-91) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:07:07,765 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-91) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:07:07,765 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-91) spm vds is non responsive, stopping spm selection. >2014-01-09 17:07:09,577 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:07:12,586 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:07:15,596 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:07:18,609 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:07:21,619 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:07:26,629 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:07:27,786 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-64) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:07:27,789 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-64) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:07:27,791 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-64) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:07:27,792 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-64) spm vds is non responsive, stopping spm selection. >2014-01-09 17:07:31,641 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:07:36,652 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:07:38,180 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp--127.0.0.1-8702-11) Running command: LoginUserCommand internal: false. >2014-01-09 17:07:38,184 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2014-01-09 17:07:41,664 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:07:46,676 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:07:47,831 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-31) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:07:47,833 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-31) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:07:47,835 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-31) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:07:47,836 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-31) spm vds is non responsive, stopping spm selection. >2014-01-09 17:07:51,688 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-43) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:07:56,700 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-27) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:08:01,712 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-21) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:08:02,053 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-8) Lock Acquired to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 17:08:02,106 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-8) CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,VM_CANNOT_RUN_FROM_CD_WITHOUT_ACTIVE_STORAGE_DOMAIN_ISO >2014-01-09 17:08:02,107 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-8) Lock freed to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 17:08:02,116 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp--127.0.0.1-8702-8) Operation Failed: [Cannot run VM with attached CD without an active ISO domain. >Alternatives: >-Attach an ISO Domain (to enable CD operations). >-Change the boot sequence, using the Edit VM command (Boot Sub-Tab). >-Select a different boot device (using the 'Run Once' command).] >2014-01-09 17:08:06,725 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:08:07,869 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:08:07,870 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:08:07,872 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:08:07,873 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) spm vds is non responsive, stopping spm selection. >2014-01-09 17:08:11,737 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:08:13,704 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-26) Lock Acquired to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 17:08:13,722 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-26) CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,VM_CANNOT_RUN_FROM_CD_WITHOUT_ACTIVE_STORAGE_DOMAIN_ISO >2014-01-09 17:08:13,723 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-26) Lock freed to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 17:08:13,729 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp--127.0.0.1-8702-26) Operation Failed: [Cannot run VM with attached CD without an active ISO domain. >Alternatives: >-Attach an ISO Domain (to enable CD operations). >-Change the boot sequence, using the Edit VM command (Boot Sub-Tab). >-Select a different boot device (using the 'Run Once' command).] >2014-01-09 17:08:16,747 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:08:21,759 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-57) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:08:26,771 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:08:27,907 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:08:27,909 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:08:27,911 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:08:27,912 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) spm vds is non responsive, stopping spm selection. >2014-01-09 17:08:31,783 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:08:36,797 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:08:41,808 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-89) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:08:46,818 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:08:47,945 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:08:47,947 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:08:47,949 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:08:47,950 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) spm vds is non responsive, stopping spm selection. >2014-01-09 17:08:51,830 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:08:56,841 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:09:01,851 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:09:02,410 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-19) Lock Acquired to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 17:09:02,428 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-19) CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,VM_CANNOT_RUN_FROM_CD_WITHOUT_ACTIVE_STORAGE_DOMAIN_ISO >2014-01-09 17:09:02,428 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-19) Lock freed to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 17:09:02,433 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp--127.0.0.1-8702-19) Operation Failed: [Cannot run VM with attached CD without an active ISO domain. >Alternatives: >-Attach an ISO Domain (to enable CD operations). >-Change the boot sequence, using the Edit VM command (Boot Sub-Tab). >-Select a different boot device (using the 'Run Once' command).] >2014-01-09 17:09:06,863 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:09:07,973 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-79) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:09:07,976 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-79) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:09:07,978 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-79) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:09:07,979 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-79) spm vds is non responsive, stopping spm selection. >2014-01-09 17:09:11,875 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:09:16,887 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:09:21,899 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-29) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:09:26,911 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:09:28,009 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:09:28,011 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:09:28,014 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:09:28,015 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) spm vds is non responsive, stopping spm selection. >2014-01-09 17:09:31,922 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:09:36,934 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:09:41,944 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:09:46,956 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-56) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:09:48,042 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:09:48,044 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:09:48,046 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:09:48,047 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) spm vds is non responsive, stopping spm selection. >2014-01-09 17:09:51,968 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:09:56,980 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:10:01,993 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:10:07,006 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:10:08,079 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-35) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:10:08,081 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-35) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:10:08,083 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-35) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:10:08,084 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-35) spm vds is non responsive, stopping spm selection. >2014-01-09 17:10:12,016 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-33) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:10:17,029 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:10:22,041 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:10:27,052 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-57) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:10:28,112 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-40) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:10:28,114 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-40) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:10:28,116 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-40) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:10:28,117 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-40) spm vds is non responsive, stopping spm selection. >2014-01-09 17:10:31,902 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-2) Lock Acquired to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 17:10:31,949 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-2) CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,VM_CANNOT_RUN_FROM_CD_WITHOUT_ACTIVE_STORAGE_DOMAIN_ISO >2014-01-09 17:10:31,950 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-2) Lock freed to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 17:10:31,954 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp--127.0.0.1-8702-2) Operation Failed: [Cannot run VM with attached CD without an active ISO domain. >Alternatives: >-Attach an ISO Domain (to enable CD operations). >-Change the boot sequence, using the Edit VM command (Boot Sub-Tab). >-Select a different boot device (using the 'Run Once' command).] >2014-01-09 17:10:32,064 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:10:33,379 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-24) Lock Acquired to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 17:10:33,396 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-24) CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,VM_CANNOT_RUN_FROM_CD_WITHOUT_ACTIVE_STORAGE_DOMAIN_ISO >2014-01-09 17:10:33,397 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-24) Lock freed to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 17:10:33,401 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp--127.0.0.1-8702-24) Operation Failed: [Cannot run VM with attached CD without an active ISO domain. >Alternatives: >-Attach an ISO Domain (to enable CD operations). >-Change the boot sequence, using the Edit VM command (Boot Sub-Tab). >-Select a different boot device (using the 'Run Once' command).] >2014-01-09 17:10:37,078 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:10:42,088 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:10:47,098 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:10:48,149 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-22) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:10:48,151 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-22) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:10:48,154 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-22) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:10:48,154 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-22) spm vds is non responsive, stopping spm selection. >2014-01-09 17:10:52,109 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:10:57,120 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:11:02,131 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:11:07,142 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:11:08,187 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-90) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:11:08,189 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-90) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:11:08,191 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-90) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:11:08,192 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-90) spm vds is non responsive, stopping spm selection. >2014-01-09 17:11:12,155 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:11:17,169 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:11:22,180 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:11:27,192 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:11:28,224 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-9) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:11:28,227 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-9) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:11:28,229 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-9) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:11:28,230 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-9) spm vds is non responsive, stopping spm selection. >2014-01-09 17:11:32,204 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:11:37,215 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:11:42,227 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:11:47,237 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:11:48,262 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-11) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:11:48,264 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-11) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:11:48,267 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-11) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:11:48,267 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-11) spm vds is non responsive, stopping spm selection. >2014-01-09 17:11:52,249 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:11:57,260 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-43) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:12:02,271 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-21) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:12:07,284 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:12:08,292 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-53) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:12:08,294 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-53) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:12:08,296 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-53) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:12:08,297 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-53) spm vds is non responsive, stopping spm selection. >2014-01-09 17:12:12,295 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:12:17,307 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-33) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:12:22,318 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:12:27,329 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:12:28,324 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-37) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:12:28,326 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-37) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:12:28,328 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-37) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:12:28,329 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-37) spm vds is non responsive, stopping spm selection. >2014-01-09 17:12:32,342 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:12:37,357 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:12:42,369 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:12:47,381 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:12:48,361 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-68) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:12:48,362 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-68) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:12:48,365 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-68) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:12:48,366 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-68) spm vds is non responsive, stopping spm selection. >2014-01-09 17:12:52,393 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:12:57,404 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:13:02,415 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:13:07,426 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:13:08,394 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-89) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:13:08,396 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-89) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:13:08,399 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-89) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:13:08,399 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-89) spm vds is non responsive, stopping spm selection. >2014-01-09 17:13:12,437 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-88) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:13:17,448 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:13:22,460 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:13:27,471 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:13:28,433 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:13:28,435 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:13:28,437 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:13:28,438 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) spm vds is non responsive, stopping spm selection. >2014-01-09 17:13:32,482 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:13:37,494 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:13:42,506 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:13:47,517 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-56) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:13:48,466 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:13:48,468 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:13:48,470 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:13:48,470 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-78) spm vds is non responsive, stopping spm selection. >2014-01-09 17:13:52,528 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:13:57,539 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:14:02,550 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:14:07,561 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:14:08,499 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-19) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:14:08,502 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-19) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:14:08,504 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-19) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:14:08,505 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-19) spm vds is non responsive, stopping spm selection. >2014-01-09 17:14:12,572 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:14:17,431 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-25) Lock Acquired to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 17:14:17,449 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-25) CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,VM_CANNOT_RUN_FROM_CD_WITHOUT_ACTIVE_STORAGE_DOMAIN_ISO >2014-01-09 17:14:17,450 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-25) Lock freed to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 17:14:17,456 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp--127.0.0.1-8702-25) Operation Failed: [Cannot run VM with attached CD without an active ISO domain. >Alternatives: >-Attach an ISO Domain (to enable CD operations). >-Change the boot sequence, using the Edit VM command (Boot Sub-Tab). >-Select a different boot device (using the 'Run Once' command).] >2014-01-09 17:14:17,584 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:14:22,594 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:14:27,606 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-57) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:14:28,524 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-43) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:14:28,525 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-43) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:14:28,528 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-43) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:14:28,528 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-43) spm vds is non responsive, stopping spm selection. >2014-01-09 17:14:31,248 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-20) Lock Acquired to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 17:14:31,266 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-20) CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,VM_CANNOT_RUN_FROM_CD_WITHOUT_ACTIVE_STORAGE_DOMAIN_ISO >2014-01-09 17:14:31,267 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-20) Lock freed to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 17:14:31,272 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp--127.0.0.1-8702-20) Operation Failed: [Cannot run VM with attached CD without an active ISO domain. >Alternatives: >-Attach an ISO Domain (to enable CD operations). >-Change the boot sequence, using the Edit VM command (Boot Sub-Tab). >-Select a different boot device (using the 'Run Once' command).] >2014-01-09 17:14:32,617 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:14:37,631 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:14:42,642 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:14:47,652 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:14:48,547 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-76) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:14:48,549 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-76) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:14:48,551 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-76) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:14:48,552 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-76) spm vds is non responsive, stopping spm selection. >2014-01-09 17:14:52,663 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:14:57,272 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-01-09 17:15:02,283 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:15:07,295 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:15:08,585 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-86) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:15:08,587 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-86) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:15:08,589 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-86) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:15:08,590 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-86) spm vds is non responsive, stopping spm selection. >2014-01-09 17:15:12,307 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-89) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:15:17,319 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:15:22,330 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:15:27,342 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:15:28,616 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-81) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:15:28,618 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-81) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:15:28,620 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-81) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:15:28,621 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-81) spm vds is non responsive, stopping spm selection. >2014-01-09 17:15:32,354 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:15:37,365 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:15:42,375 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:15:47,385 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:15:48,640 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-23) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:15:48,643 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-23) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:15:48,645 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-23) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:15:48,646 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-23) spm vds is non responsive, stopping spm selection. >2014-01-09 17:15:52,396 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:15:57,409 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:16:02,420 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:16:07,431 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:16:08,678 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-66) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:16:08,679 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-66) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:16:08,682 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-66) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:16:08,683 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-66) spm vds is non responsive, stopping spm selection. >2014-01-09 17:16:12,442 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:16:17,454 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:16:22,465 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:16:27,476 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:16:28,717 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-13) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:16:28,718 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-13) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:16:28,720 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-13) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:16:28,720 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-13) spm vds is non responsive, stopping spm selection. >2014-01-09 17:16:32,487 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:16:37,498 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-55) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:16:42,511 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:16:47,523 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:16:48,751 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-59) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:16:48,753 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-59) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:16:48,755 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-59) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:16:48,756 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-59) spm vds is non responsive, stopping spm selection. >2014-01-09 17:16:52,534 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:16:57,545 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:17:02,556 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-77) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:17:07,568 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:17:08,789 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-69) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:17:08,791 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-69) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:17:08,793 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-69) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:17:08,793 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-69) spm vds is non responsive, stopping spm selection. >2014-01-09 17:17:12,579 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-88) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:17:17,590 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:17:22,601 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:17:27,612 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-81) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:17:28,821 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:17:28,823 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:17:28,825 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:17:28,826 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-8) spm vds is non responsive, stopping spm selection. >2014-01-09 17:17:32,622 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:17:37,633 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:17:42,644 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:17:47,655 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:17:48,857 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-20) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:17:48,859 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-20) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:17:48,862 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-20) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:17:48,862 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-20) spm vds is non responsive, stopping spm selection. >2014-01-09 17:17:52,665 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:17:57,676 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:18:02,687 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:18:07,698 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:18:08,893 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-10) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:18:08,895 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-10) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:18:08,897 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-10) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:18:08,898 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-10) spm vds is non responsive, stopping spm selection. >2014-01-09 17:18:12,709 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:18:17,721 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-33) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:18:22,733 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:18:27,744 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:18:28,921 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:18:28,923 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:18:28,925 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:18:28,926 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) spm vds is non responsive, stopping spm selection. >2014-01-09 17:18:32,756 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:18:37,768 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:18:42,780 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:18:47,792 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:18:48,939 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:18:48,941 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:18:48,943 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:18:48,944 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-50) spm vds is non responsive, stopping spm selection. >2014-01-09 17:18:52,803 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:18:57,814 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:19:02,825 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:19:07,836 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:19:08,976 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-55) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:19:08,978 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-55) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:19:08,981 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-55) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:19:08,981 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-55) spm vds is non responsive, stopping spm selection. >2014-01-09 17:19:12,848 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:19:17,859 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:19:22,870 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:19:27,881 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:19:29,014 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-99) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:19:29,015 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-99) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:19:29,018 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-99) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:19:29,018 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-99) spm vds is non responsive, stopping spm selection. >2014-01-09 17:19:32,892 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:19:37,903 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:19:42,912 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:19:47,924 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:19:49,052 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-16) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:19:49,054 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-16) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:19:49,056 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-16) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:19:49,057 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-16) spm vds is non responsive, stopping spm selection. >2014-01-09 17:19:52,934 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:19:57,945 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:20:02,955 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:20:07,966 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:20:09,089 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-3) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:20:09,091 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-3) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:20:09,093 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-3) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:20:09,094 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-3) spm vds is non responsive, stopping spm selection. >2014-01-09 17:20:12,977 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:20:17,988 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:20:22,999 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:20:28,011 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-43) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:20:29,128 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:20:29,130 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:20:29,132 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:20:29,133 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-61) spm vds is non responsive, stopping spm selection. >2014-01-09 17:20:33,023 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-87) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:20:38,034 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-27) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:20:43,047 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:20:48,058 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:20:49,166 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:20:49,168 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:20:49,170 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:20:49,171 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) spm vds is non responsive, stopping spm selection. >2014-01-09 17:20:53,069 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:20:57,089 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:21:00,097 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:21:03,107 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:21:06,117 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-55) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:21:09,126 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:21:09,204 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-80) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:21:09,205 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-80) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:21:09,208 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-80) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:21:09,208 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-80) spm vds is non responsive, stopping spm selection. >2014-01-09 17:21:12,135 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:21:15,187 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-88) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:21:18,197 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:21:21,207 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-29) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:21:24,216 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:21:27,228 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:21:29,236 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-65) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:21:29,238 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-65) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:21:29,240 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-65) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:21:29,241 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-65) spm vds is non responsive, stopping spm selection. >2014-01-09 17:21:30,236 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:21:33,245 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-56) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:21:36,254 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:21:39,264 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:21:42,273 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:21:45,283 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:21:48,291 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:21:49,257 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-91) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:21:49,259 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-91) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:21:49,261 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-91) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:21:49,262 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-91) spm vds is non responsive, stopping spm selection. >2014-01-09 17:21:51,300 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:21:54,309 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:21:57,318 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:22:00,327 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:22:03,336 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:22:06,345 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:22:09,289 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:22:09,291 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:22:09,293 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:22:09,294 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) spm vds is non responsive, stopping spm selection. >2014-01-09 17:22:09,354 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:22:12,363 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:22:14,211 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp--127.0.0.1-8702-20) Running command: LoginUserCommand internal: false. >2014-01-09 17:22:14,216 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-20) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2014-01-09 17:22:15,372 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:22:18,381 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:22:21,390 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:22:24,400 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:22:27,409 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:22:29,326 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-10) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:22:29,328 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-10) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:22:29,330 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-10) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:22:29,331 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-10) spm vds is non responsive, stopping spm selection. >2014-01-09 17:22:30,418 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:22:33,428 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:22:36,436 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:22:36,891 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-3) Lock Acquired to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 17:22:36,905 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-3) CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,VM_CANNOT_RUN_FROM_CD_WITHOUT_ACTIVE_STORAGE_DOMAIN_ISO >2014-01-09 17:22:36,906 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-3) Lock freed to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 17:22:36,909 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp--127.0.0.1-8702-3) Operation Failed: [Cannot run VM with attached CD without an active ISO domain. >Alternatives: >-Attach an ISO Domain (to enable CD operations). >-Change the boot sequence, using the Edit VM command (Boot Sub-Tab). >-Select a different boot device (using the 'Run Once' command).] >2014-01-09 17:22:39,446 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:22:42,456 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:22:45,465 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:22:48,475 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:22:49,358 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:22:49,359 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:22:49,362 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:22:49,362 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-41) spm vds is non responsive, stopping spm selection. >2014-01-09 17:22:51,486 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:22:54,497 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:22:57,505 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-88) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:23:00,514 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:23:03,523 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:23:06,531 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:23:09,396 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:23:09,398 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:23:09,400 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:23:09,401 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-47) spm vds is non responsive, stopping spm selection. >2014-01-09 17:23:09,541 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:23:12,550 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:23:15,559 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:23:18,568 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:23:21,577 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:23:24,586 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:23:27,595 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:23:29,428 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-28) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:23:29,430 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-28) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:23:29,432 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-28) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:23:29,433 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-28) spm vds is non responsive, stopping spm selection. >2014-01-09 17:23:30,605 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:23:33,614 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:23:36,623 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:23:39,631 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:23:42,641 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:23:45,650 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:23:48,659 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:23:49,447 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-36) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:23:49,449 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-36) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:23:49,451 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-36) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:23:49,452 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-36) spm vds is non responsive, stopping spm selection. >2014-01-09 17:23:51,668 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:23:54,678 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:23:57,688 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-57) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:24:00,696 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:24:03,706 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:24:06,716 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:24:09,483 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:24:09,485 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:24:09,487 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:24:09,488 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) spm vds is non responsive, stopping spm selection. >2014-01-09 17:24:09,725 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:24:12,735 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:24:15,745 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:24:18,755 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:24:21,764 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:24:24,773 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:24:27,782 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:24:29,519 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-52) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:24:29,520 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-52) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:24:29,523 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-52) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:24:29,524 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-52) spm vds is non responsive, stopping spm selection. >2014-01-09 17:24:30,791 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:24:33,801 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-55) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:24:36,811 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:24:39,819 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:24:42,829 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:24:45,838 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:24:48,847 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:24:49,547 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:24:49,549 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:24:49,552 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:24:49,552 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) spm vds is non responsive, stopping spm selection. >2014-01-09 17:24:51,857 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:24:54,866 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:24:57,876 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:25:00,885 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:25:05,897 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:25:09,585 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-7) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:25:09,587 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-7) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:25:09,589 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-7) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:25:09,590 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-7) spm vds is non responsive, stopping spm selection. >2014-01-09 17:25:10,908 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:25:15,920 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:25:20,931 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:25:25,943 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:25:29,622 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-89) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:25:29,624 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-89) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:25:29,626 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-89) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:25:29,627 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-89) spm vds is non responsive, stopping spm selection. >2014-01-09 17:25:30,954 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:25:35,965 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:25:40,977 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:25:45,988 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:25:49,660 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-11) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:25:49,662 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-11) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:25:49,664 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-11) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:25:49,665 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-11) spm vds is non responsive, stopping spm selection. >2014-01-09 17:25:50,999 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:25:56,010 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:26:01,021 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:26:06,032 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:26:09,687 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-51) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:26:09,689 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-51) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:26:09,691 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-51) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:26:09,692 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-51) spm vds is non responsive, stopping spm selection. >2014-01-09 17:26:11,043 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:26:16,053 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:26:21,064 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:26:26,074 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:26:29,723 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:26:29,725 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:26:29,728 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:26:29,728 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-100) spm vds is non responsive, stopping spm selection. >2014-01-09 17:26:31,086 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:26:36,096 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-88) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:26:41,109 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-29) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:26:46,120 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-43) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:26:49,761 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-30) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:26:49,762 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-30) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:26:49,765 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-30) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:26:49,765 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-30) spm vds is non responsive, stopping spm selection. >2014-01-09 17:26:51,132 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:26:56,144 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:27:01,155 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:27:06,166 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:27:09,797 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:27:09,799 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:27:09,801 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:27:09,801 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-84) spm vds is non responsive, stopping spm selection. >2014-01-09 17:27:11,177 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:27:16,188 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:27:21,200 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:27:26,211 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:27:29,833 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-49) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:27:29,835 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-49) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:27:29,837 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-49) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:27:29,838 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-49) spm vds is non responsive, stopping spm selection. >2014-01-09 17:27:31,222 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:27:36,233 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:27:41,243 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:27:46,255 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:27:49,870 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-65) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:27:49,872 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-65) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:27:49,874 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-65) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:27:49,874 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-65) spm vds is non responsive, stopping spm selection. >2014-01-09 17:27:51,265 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:27:56,276 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:28:01,287 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:28:06,298 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:28:09,906 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:28:09,908 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:28:09,910 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:28:09,911 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-73) spm vds is non responsive, stopping spm selection. >2014-01-09 17:28:11,310 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:28:16,321 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:28:21,333 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:28:26,343 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:28:29,944 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:28:29,945 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:28:29,948 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:28:29,948 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) spm vds is non responsive, stopping spm selection. >2014-01-09 17:28:31,352 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:28:36,364 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:28:41,377 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-88) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:28:46,388 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-81) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:28:49,989 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:28:49,991 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:28:49,994 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:28:49,994 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-45) spm vds is non responsive, stopping spm selection. >2014-01-09 17:28:51,400 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 17:28:54,410 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:28:57,419 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:29:00,427 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:29:03,436 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:29:06,445 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:29:09,453 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:29:10,027 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:29:10,029 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:29:10,031 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:29:10,032 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-33) spm vds is non responsive, stopping spm selection. >2014-01-09 17:29:12,463 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:29:15,472 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:29:18,482 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-8) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:29:21,491 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:29:24,500 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-56) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:29:27,510 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:29:30,072 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-14) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:29:30,074 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-14) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:29:30,076 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-14) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:29:30,076 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-14) spm vds is non responsive, stopping spm selection. >2014-01-09 17:29:30,521 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:29:33,530 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-58) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:29:36,538 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:29:39,547 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:29:42,557 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:29:45,567 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-37) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:29:48,577 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-87) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:29:50,109 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-54) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:29:50,111 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-54) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:29:50,113 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-54) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:29:50,114 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-54) spm vds is non responsive, stopping spm selection. >2014-01-09 17:29:51,586 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:29:54,595 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:29:57,604 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:30:00,613 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:30:03,621 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:30:06,630 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:30:09,639 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:30:10,143 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:30:10,145 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:30:10,147 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:30:10,148 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-21) spm vds is non responsive, stopping spm selection. >2014-01-09 17:30:12,648 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:30:15,657 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-41) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:30:18,667 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-55) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:30:21,677 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:30:24,686 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:30:27,695 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-29) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:30:30,174 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-96) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:30:30,176 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-96) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:30:30,178 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-96) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:30:30,179 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-96) spm vds is non responsive, stopping spm selection. >2014-01-09 17:30:30,704 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:30:33,714 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:30:36,723 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:30:39,732 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:30:42,747 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:30:45,756 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:30:48,765 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:30:50,210 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:30:50,212 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:30:50,214 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:30:50,214 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-46) spm vds is non responsive, stopping spm selection. >2014-01-09 17:30:51,774 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:30:54,786 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:30:57,438 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp--127.0.0.1-8702-5) Running command: LoginUserCommand internal: false. >2014-01-09 17:30:57,443 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2014-01-09 17:30:57,795 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:31:00,805 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:31:03,814 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:31:06,822 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:31:08,180 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-26) Lock Acquired to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 17:31:08,197 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-26) CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,VM_CANNOT_RUN_FROM_CD_WITHOUT_ACTIVE_STORAGE_DOMAIN_ISO >2014-01-09 17:31:08,197 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-26) Lock freed to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 17:31:08,202 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp--127.0.0.1-8702-26) Operation Failed: [Cannot run VM with attached CD without an active ISO domain. >Alternatives: >-Attach an ISO Domain (to enable CD operations). >-Change the boot sequence, using the Edit VM command (Boot Sub-Tab). >-Select a different boot device (using the 'Run Once' command).] >2014-01-09 17:31:09,832 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-01-09 17:31:10,246 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:31:10,249 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:31:10,251 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:31:10,252 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-18) spm vds is non responsive, stopping spm selection. >2014-01-09 17:31:15,327 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-17) Lock Acquired to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 17:31:15,371 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-17) CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,VM_CANNOT_RUN_FROM_CD_WITHOUT_ACTIVE_STORAGE_DOMAIN_ISO >2014-01-09 17:31:15,372 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-17) Lock freed to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 17:31:15,376 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp--127.0.0.1-8702-17) Operation Failed: [Cannot run VM with attached CD without an active ISO domain. >Alternatives: >-Attach an ISO Domain (to enable CD operations). >-Change the boot sequence, using the Edit VM command (Boot Sub-Tab). >-Select a different boot device (using the 'Run Once' command).] >2014-01-09 17:31:21,797 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-8) Lock Acquired to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 17:31:21,807 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-8) CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,ACTION_TYPE_FAILED_VM_IS_DURING_SNAPSHOT >2014-01-09 17:31:21,808 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-8) Lock freed to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 17:31:21,812 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp--127.0.0.1-8702-8) Operation Failed: [Cannot run VM. The VM is performing an operation on a Snapshot. Please wait for the operation to finish, and try again.] >2014-01-09 17:31:30,283 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:31:30,285 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:31:30,287 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:31:30,288 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-62) spm vds is non responsive, stopping spm selection. >2014-01-09 17:31:32,328 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-18) Lock Acquired to object EngineLock [exclusiveLocks= key: 565c5082-b8a3-4946-8ae2-6d3330c8ef9b value: VM >, sharedLocks= ] >2014-01-09 17:31:32,338 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-18) CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,ACTION_TYPE_FAILED_VM_IS_LOCKED >2014-01-09 17:31:32,339 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-18) Lock freed to object EngineLock [exclusiveLocks= key: 565c5082-b8a3-4946-8ae2-6d3330c8ef9b value: VM >, sharedLocks= ] >2014-01-09 17:31:32,342 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp--127.0.0.1-8702-18) Operation Failed: [Cannot run VM: VM is locked. Please try again in a few minutes.] >2014-01-09 17:31:33,726 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-78) START, GetHardwareInfoVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01]), log id: 34f78c95 >2014-01-09 17:31:50,321 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:31:50,322 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:31:50,325 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:31:50,325 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-93) spm vds is non responsive, stopping spm selection. >2014-01-09 17:31:53,934 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-78) FINISH, GetHardwareInfoVDSCommand, log id: 34f78c95 >2014-01-09 17:31:53,947 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-78) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host ovirt01. >2014-01-09 17:32:04,010 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-31) Lock Acquired to object EngineLock [exclusiveLocks= key: 872c3dea-6984-48fa-99c3-354fd367c3f9 value: VM >, sharedLocks= ] >2014-01-09 17:32:04,028 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-31) CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,VM_CANNOT_RUN_FROM_CD_WITHOUT_ACTIVE_STORAGE_DOMAIN_ISO >2014-01-09 17:32:04,029 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-31) Lock freed to object EngineLock [exclusiveLocks= key: 872c3dea-6984-48fa-99c3-354fd367c3f9 value: VM >, sharedLocks= ] >2014-01-09 17:32:04,033 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (ajp--127.0.0.1-8702-31) Operation Failed: [Cannot run VM with attached CD without an active ISO domain. >Alternatives: >-Attach an ISO Domain (to enable CD operations). >-Change the boot sequence, using the Edit VM command (Boot Sub-Tab). >-Select a different boot device (using the 'Run Once' command).] >2014-01-09 17:32:10,351 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-54) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:32:10,353 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-54) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:32:10,356 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-54) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:32:10,356 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-54) spm vds is non responsive, stopping spm selection. >2014-01-09 17:32:13,977 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-78) vm DC-02 running in db and not running in vds - add to rerun treatment. vds ovirt01 >2014-01-09 17:32:13,978 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-78) vm DC-01 running in db and not running in vds - add to rerun treatment. vds ovirt01 >2014-01-09 17:32:13,978 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-78) vm External_Email running in db and not running in vds - add to rerun treatment. vds ovirt01 >2014-01-09 17:32:13,979 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-78) Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 17:32:13,982 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-78) Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 17:32:14,057 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-78) START, ConnectStorageServerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = ISCSI, connectionList = [{ id: 623aa506-12d1-4437-8e81-f39b2317f3dd, connection: 192.168.101.101, iqn: iqn.2013-11.local.cytechservices.n16000pro:iscsi.hosts-tgt00.raid0, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 35086193 >2014-01-09 17:32:30,389 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-10) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 17:32:30,391 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-10) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 17:32:30,393 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-10) SPM selection - vds seems as spm ovirt01 >2014-01-09 17:32:30,394 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-10) spm vds is non responsive, stopping spm selection. >2014-01-09 18:33:47,223 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-01-09 18:33:47,251 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Loaded file "/etc/ovirt-engine/engine.conf". >2014-01-09 18:33:47,252 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-01-09 18:33:47,252 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-java.conf". >2014-01-09 18:33:47,253 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-01-09 18:33:47,253 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-01-09 18:33:47,254 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-01-09 18:33:47,259 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_AJP_ENABLED" is "true". >2014-01-09 18:33:47,260 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_AJP_PORT" is "8702". >2014-01-09 18:33:47,260 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_APPS" is "engine.ear". >2014-01-09 18:33:47,261 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine". >2014-01-09 18:33:47,266 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-01-09 18:33:47,266 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-01-09 18:33:47,267 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-01-09 18:33:47,267 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-01-09 18:33:47,268 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_DB_HOST" is "localhost". >2014-01-09 18:33:47,268 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-01-09 18:33:47,269 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-01-09 18:33:47,269 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-01-09 18:33:47,269 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_DB_PORT" is "5432". >2014-01-09 18:33:47,270 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_DB_SECURED" is "False". >2014-01-09 18:33:47,270 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-01-09 18:33:47,271 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_DB_SSL" is "false". >2014-01-09 18:33:47,271 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-01-09 18:33:47,271 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_DB_USER" is "engine". >2014-01-09 18:33:47,272 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_DEBUG_ADDRESS" is "". >2014-01-09 18:33:47,272 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine". >2014-01-09 18:33:47,273 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_ETC" is "/etc/ovirt-engine". >2014-01-09 18:33:47,273 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_FQDN" is "ovirt00.mgmt.cytechservices.com". >2014-01-09 18:33:47,274 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_GROUP" is "ovirt". >2014-01-09 18:33:47,280 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-01-09 18:33:47,280 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-01-09 18:33:47,281 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_HTTPS_ENABLED" is "false". >2014-01-09 18:33:47,281 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_HTTPS_PORT" is "None". >2014-01-09 18:33:47,282 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-01-09 18:33:47,282 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_HTTP_ENABLED" is "false". >2014-01-09 18:33:47,282 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_HTTP_PORT" is "None". >2014-01-09 18:33:47,283 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"". >2014-01-09 18:33:47,283 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine". >2014-01-09 18:33:47,284 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_LOG_TO_CONSOLE" is "false". >2014-01-09 18:33:47,284 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_MANUAL" is "/usr/share/ovirt-engine/manual". >2014-01-09 18:33:47,284 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_PERM_MAX" is "256m". >2014-01-09 18:33:47,291 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_PERM_MIN" is "256m". >2014-01-09 18:33:47,293 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine". >2014-01-09 18:33:47,294 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem". >2014-01-09 18:33:47,294 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer". >2014-01-09 18:33:47,295 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12". >2014-01-09 18:33:47,295 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-01-09 18:33:47,296 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-01-09 18:33:47,296 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore". >2014-01-09 18:33:47,296 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-01-09 18:33:47,297 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false". >2014-01-09 18:33:47,297 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_PROXY_ENABLED" is "true". >2014-01-09 18:33:47,298 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-01-09 18:33:47,298 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-01-09 18:33:47,299 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-01-09 18:33:47,299 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_STOP_TIME" is "10". >2014-01-09 18:33:47,299 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine". >2014-01-09 18:33:47,300 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up". >2014-01-09 18:33:47,300 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_USER" is "ovirt". >2014-01-09 18:33:47,300 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine". >2014-01-09 18:33:47,301 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine". >2014-01-09 18:33:47,301 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-01-09 18:33:47,301 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "JAVA_HOME" is "/usr/lib/jvm/jre-1.7.0-openjdk.x86_64". >2014-01-09 18:33:47,302 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "JBOSS_HOME" is "/usr/share/jboss-as". >2014-01-09 18:33:47,302 INFO [org.ovirt.engine.core.utils.LocalConfig] (MSC service thread 1-4) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-01-09 18:33:47,824 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-2) Start initializing Backend >2014-01-09 18:33:48,117 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-2) Running ovirt-engine 3.3.2-1.el6 >2014-01-09 18:33:48,119 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-2) Start initializing dictionaries >2014-01-09 18:33:48,122 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (MSC service thread 1-2) Finished initializing dictionaries >2014-01-09 18:33:48,123 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-2) Start initializing AuditLogCleanupManager >2014-01-09 18:33:48,123 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-2) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-01-09 18:33:48,142 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (MSC service thread 1-2) Finished initializing AuditLogCleanupManager >2014-01-09 18:33:48,145 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-2) Start initializing TagsDirector >2014-01-09 18:33:48,155 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-2) Tag root added to tree >2014-01-09 18:33:48,168 INFO [org.ovirt.engine.core.bll.TagsDirector] (MSC service thread 1-2) Finished initializing TagsDirector >2014-01-09 18:33:48,170 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-2) Start initializing IsoDomainListSyncronizer >2014-01-09 18:33:48,176 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (MSC service thread 1-2) Finished initializing IsoDomainListSyncronizer >2014-01-09 18:33:48,185 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (MSC service thread 1-2) Loaded file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-01-09 18:33:48,304 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-2) Completed initializing handlers >2014-01-09 18:33:48,306 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-2) Start initializing ErrorTranslatorImpl >2014-01-09 18:33:48,315 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-2) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-01-09 18:33:48,317 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-2) Finished initializing ErrorTranslatorImpl >2014-01-09 18:33:48,317 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-2) Start initializing ErrorTranslatorImpl >2014-01-09 18:33:48,318 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (MSC service thread 1-2) Finished initializing ErrorTranslatorImpl >2014-01-09 18:33:48,318 INFO [org.ovirt.engine.core.bll.Backend] (MSC service thread 1-2) Mark incomplete jobs as UNKNOWN >2014-01-09 18:33:48,344 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-2) Start initializing JobRepositoryCleanupManager >2014-01-09 18:33:48,345 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (MSC service thread 1-2) Finished initializing JobRepositoryCleanupManager >2014-01-09 18:33:48,346 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-2) Start initializing AutoRecoveryManager >2014-01-09 18:33:48,348 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (MSC service thread 1-2) Finished initializing AutoRecoveryManager >2014-01-09 18:33:48,350 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-2) Start initializing ExecutionMessageDirector >2014-01-09 18:33:48,351 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (MSC service thread 1-2) Finished initializing ExecutionMessageDirector >2014-01-09 18:33:48,370 INFO [org.ovirt.engine.core.bll.adbroker.UsersDomainsCacheManagerService] (MSC service thread 1-4) Start initializing UsersDomainsCacheManagerService >2014-01-09 18:33:48,392 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (MSC service thread 1-6) Initialization of AsyncTaskManager completed successfully. >2014-01-09 18:33:48,394 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-6) Start initializing ResourceManager >2014-01-09 18:33:48,475 ERROR [org.ovirt.engine.core.utils.dns.DnsSRVLocator] (MSC service thread 1-4) Error: could not find DNS SRV record name: _ldap._tcp.cytechservices.local. >Exception message is: DNS name not found [response code 3] >Possible causes: missing DNS entries in the DNS server or DNS resolving issues from engine-core machine. >Please Ensure correct DNS entries exist in the DNS server and ensure the DNS server is reachable from the engine-core machine. >2014-01-09 18:33:48,508 WARN [org.ovirt.engine.core.bll.adbroker.UsersDomainsCacheManagerService] (MSC service thread 1-4) Error in getting LDAP servers for domain cytechservices.local: Authentication Failed. Please verify the fully qualified domain name that is used for authentication is correct.. Constructing an LDAP URL based on domain name >2014-01-09 18:33:48,511 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (MSC service thread 1-4) Start initializing DbUserCacheManager >2014-01-09 18:33:48,515 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (MSC service thread 1-4) Finished initializing DbUserCacheManager >2014-01-09 18:33:48,515 INFO [org.ovirt.engine.core.bll.adbroker.UsersDomainsCacheManagerService] (MSC service thread 1-4) Finished initializing UsersDomainsCacheManagerService >2014-01-09 18:33:48,536 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-1) Start refreshing all users data >2014-01-09 18:33:48,560 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) AuditLogType: UNASSIGNED not exist in string table >2014-01-09 18:33:48,561 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-01-09 18:33:48,561 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-01-09 18:33:48,562 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-01-09 18:33:48,563 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-01-09 18:33:48,563 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) AuditLogType: DWH_STOPPED not exist in string table >2014-01-09 18:33:48,564 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) AuditLogType: DWH_STARTED not exist in string table >2014-01-09 18:33:48,564 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) AuditLogType: DWH_ERROR not exist in string table >2014-01-09 18:33:48,571 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) AuditLogType: USER_FAILED_REMOVE_VM not have severity. Assumed Normal >2014-01-09 18:33:48,572 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) AuditLogType: USER_ATTACH_DISK_TO_VM not have severity. Assumed Normal >2014-01-09 18:33:48,572 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) AuditLogType: USER_DETACH_DISK_FROM_VM not have severity. Assumed Normal >2014-01-09 18:33:48,573 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) AuditLogType: USER_FAILED_DETACH_DISK_FROM_VM not have severity. Assumed Normal >2014-01-09 18:33:48,573 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not have severity. Assumed Normal >2014-01-09 18:33:48,574 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) AuditLogType: GLUSTER_VOLUME_OPTION_CHANGED_FROM_CLI not have severity. Assumed Normal >2014-01-09 18:33:48,574 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) AuditLogType: GLUSTER_SERVICES_LIST_NOT_FETCHED not have severity. Assumed Normal >2014-01-09 18:33:48,575 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) AuditLogType: GLUSTER_VOLUME_BRICK_ADDED not have severity. Assumed Normal >2014-01-09 18:33:48,575 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) AuditLogType: USER_EXTEND_DISK_SIZE_UPDATE_VM_FAILURE not have severity. Assumed Normal >2014-01-09 18:33:48,576 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) AuditLogType: VM_MIGRATION_START_SYSTEM_INITIATED not have severity. Assumed Normal >2014-01-09 18:33:48,577 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not have severity. Assumed Normal >2014-01-09 18:33:48,577 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) AuditLogType: NETWORK_UPDATE_VM_INTERFACE_LINK_UP not have severity. Assumed Normal >2014-01-09 18:33:48,578 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) AuditLogType: NETWORK_UPDATE_VM_INTERFACE_LINK_DOWN not have severity. Assumed Normal >2014-01-09 18:33:48,579 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) AuditLogType: USER_UPDATED_MOM_POLICIES not have severity. Assumed Normal >2014-01-09 18:33:48,579 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) AuditLogType: USER_FAILED_TO_UPDATE_MOM_POLICIES not have severity. Assumed Normal >2014-01-09 18:33:48,952 ERROR [org.ovirt.engine.core.bll.adbroker.GetRootDSE] (DefaultQuartzScheduler_Worker-1) Failed to query rootDSE for LDAP server ldap://cytechservices.local:389 due to cytechservices.local:389 >2014-01-09 18:33:48,956 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-1) Failed ldap search server ldap://cytechservices.local:389 using user ryan-a@CYTECHSERVICES.LOCAL due to javax.naming.CommunicationException: cytechservices.local:389 [Root exception is java.net.UnknownHostException: cytechservices.local]. We should try the next server >2014-01-09 18:33:48,958 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-1) User Ryan not found in directory server, its status switched to InActive >2014-01-09 18:33:48,962 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM DC-02 was set to the Unknown status. >2014-01-09 18:33:48,970 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-1) User Ryan not found in directory server, its status switched to InActive >2014-01-09 18:33:48,972 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-1) User Patrick not found in directory server, its status switched to InActive >2014-01-09 18:33:48,974 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-1) User Chris not found in directory server, its status switched to InActive >2014-01-09 18:33:48,976 WARN [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-1) User Patrick not found in directory server, its status switched to InActive >2014-01-09 18:33:48,998 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM External_Email was set to the Unknown status. >2014-01-09 18:33:49,011 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (MSC service thread 1-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM DC-01 was set to the Unknown status. >2014-01-09 18:33:49,025 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-6) Entered VdsManager constructor >2014-01-09 18:33:49,040 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-6) Initialize vdsBroker (192.168.100.22,54,321) >2014-01-09 18:33:49,138 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-6) VDS bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 was added to the Resource Manager >2014-01-09 18:33:49,138 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-6) Entered VdsManager constructor >2014-01-09 18:33:49,149 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (MSC service thread 1-6) Initialize vdsBroker (192.168.100.21,54,321) >2014-01-09 18:33:49,157 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-6) VDS f2d7a8a9-e77e-4e9a-8031-23a0731407c8 was added to the Resource Manager >2014-01-09 18:33:49,184 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (MSC service thread 1-6) Finished initializing ResourceManager >2014-01-09 18:33:49,191 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (MSC service thread 1-6) Initialization of OvfDataUpdater completed successfully. >2014-01-09 18:33:49,195 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-6) Start scheduling to enable vds load balancer >2014-01-09 18:33:49,196 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-6) Finished scheduling to enable vds load balancer >2014-01-09 18:33:49,206 INFO [org.ovirt.engine.core.bll.network.MacPoolManager] (pool-6-thread-1) MacPoolManager(1480f818): Start initializing >2014-01-09 18:33:49,220 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-6) Init VM custom properties utilities >2014-01-09 18:33:49,223 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (MSC service thread 1-6) Init device custom properties utilities >2014-01-09 18:33:49,226 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-6) Initializing Scheduling manager >2014-01-09 18:33:49,246 INFO [org.ovirt.engine.core.bll.network.MacPoolManager] (pool-6-thread-1) MacPoolManager(1480f818): Finished initializing >2014-01-09 18:33:49,253 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-6) External scheduler disabled, discovery skipped >2014-01-09 18:33:49,254 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (MSC service thread 1-6) Initialized Scheduling manager >2014-01-09 18:33:49,255 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (MSC service thread 1-6) Initializing DWH Heart Beat >2014-01-09 18:33:52,566 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-4) START, GetHardwareInfoVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01]), log id: 4fdc08cd >2014-01-09 18:33:52,617 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-4) FINISH, GetHardwareInfoVDSCommand, log id: 4fdc08cd >2014-01-09 18:33:52,666 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-3) START, GetHardwareInfoVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02]), log id: 4374bc49 >2014-01-09 18:33:52,734 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-3) FINISH, GetHardwareInfoVDSCommand, log id: 4374bc49 >2014-01-09 18:33:52,763 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-4) START, GetHardwareInfoVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01]), log id: 7935848a >2014-01-09 18:33:52,789 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host ovirt02. >2014-01-09 18:33:52,824 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-4) FINISH, GetHardwareInfoVDSCommand, log id: 7935848a >2014-01-09 18:33:52,849 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host ovirt01. >2014-01-09 18:33:52,897 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) vm DC-02 running in db and not running in vds - add to rerun treatment. vds ovirt01 >2014-01-09 18:33:52,898 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) vm DC-01 running in db and not running in vds - add to rerun treatment. vds ovirt01 >2014-01-09 18:33:52,899 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) vm External_Email running in db and not running in vds - add to rerun treatment. vds ovirt01 >2014-01-09 18:33:52,987 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-3) Initializing Host: ovirt02 >2014-01-09 18:33:53,082 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-3) [3156d8f0] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 18:33:53,083 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-4) [69a684d] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 18:33:53,099 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-4) [679f817a] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 18:33:53,235 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-4) [679f817a] START, ConnectStorageServerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = ISCSI, connectionList = [{ id: 623aa506-12d1-4437-8e81-f39b2317f3dd, connection: 192.168.101.101, iqn: iqn.2013-11.local.cytechservices.n16000pro:iscsi.hosts-tgt00.raid0, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 7b9fde52 >2014-01-09 18:33:53,588 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-4) [679f817a] FINISH, ConnectStorageServerVDSCommand, return: {623aa506-12d1-4437-8e81-f39b2317f3dd=0}, log id: 7b9fde52 >2014-01-09 18:33:53,589 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-4) [679f817a] Host ovirt01 storage connection was succeeded >2014-01-09 18:33:53,592 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-4) [679f817a] START, ConnectStorageServerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 210997b7-9098-48d6-8e97-9689085aa4b0, connection: 192.168.101.101:/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: AUTO, nfsRetrans: null, nfsTimeo: null };]), log id: 288bae74 >2014-01-09 18:33:53,983 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-4) [679f817a] FINISH, ConnectStorageServerVDSCommand, return: {210997b7-9098-48d6-8e97-9689085aa4b0=0}, log id: 288bae74 >2014-01-09 18:33:53,984 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-4) [679f817a] Host ovirt01 storage connection was succeeded >2014-01-09 18:33:53,986 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-4) [679f817a] START, ConnectStorageServerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 4e5d4030-1bd2-4487-b08f-2135eba3f11f, connection: 192.168.101.101:/vms, iqn: null, vfsType: null, mountOptions: null, nfsVersion: V4, nfsRetrans: null, nfsTimeo: null };]), log id: 2ef9ac59 >2014-01-09 18:33:54,097 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-4) [679f817a] FINISH, ConnectStorageServerVDSCommand, return: {4e5d4030-1bd2-4487-b08f-2135eba3f11f=0}, log id: 2ef9ac59 >2014-01-09 18:33:54,098 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-4) [679f817a] Host ovirt01 storage connection was succeeded >2014-01-09 18:33:54,112 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-6-thread-17) START, ConnectStoragePoolVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, vds_spm_id = 1, masterDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, masterVersion = 2), log id: 317ddc8f >2014-01-09 18:33:59,251 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-9) hostFromVds::selectedVds - ovirt02, spmStatus Free, storage pool CSEnt >2014-01-09 18:33:59,262 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-9) SPM Init: could not find reported vds or not up - pool:CSEnt vds_spm_id: 1 >2014-01-09 18:33:59,275 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-9) SPM selection - vds seems as spm ovirt01 >2014-01-09 18:33:59,279 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-9) START, SpmStopVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 5c817380 >2014-01-09 18:33:59,327 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand return value > >TaskStatusListReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=654, mMessage=Not SPM: ()]] > >2014-01-09 18:33:59,329 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-9) HostName = ovirt01 >2014-01-09 18:33:59,333 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command HSMGetAllTasksStatusesVDS execution failed. Exception: IRSNonOperationalException: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM: () >2014-01-09 18:33:59,334 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-9) FINISH, SpmStopVDSCommand, log id: 5c817380 >2014-01-09 18:33:59,335 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-9) spm stop succeeded, continuing with spm selection >2014-01-09 18:33:59,342 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-9) starting spm on vds ovirt02, storage pool CSEnt, prevId 1, LVER 15 >2014-01-09 18:33:59,346 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-9) START, SpmStartVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, prevId=1, prevLVER=15, storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=true), log id: 6044a858 >2014-01-09 18:33:59,389 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-9) spmStart polling started: taskId = e3412952-bdc7-4979-8389-317ad278acda >2014-01-09 18:34:00,628 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-6-thread-17) FINISH, ConnectStoragePoolVDSCommand, log id: 317ddc8f >2014-01-09 18:34:00,685 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-4) [679f817a] START, SetMOMPolicyParametersVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8), log id: 5beeef57 >2014-01-09 18:34:00,791 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-4) [679f817a] FINISH, SetMOMPolicyParametersVDSCommand, log id: 5beeef57 >2014-01-09 18:34:00,799 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) [679f817a] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host ovirt01. >2014-01-09 18:34:00,829 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) [679f817a] Initializing Host: ovirt01 >2014-01-09 18:34:00,890 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (DefaultQuartzScheduler_Worker-4) [679f817a] Procceed on vm stop entered: 33f88d50-30eb-45c7-9850-733da17ed7a4 >2014-01-09 18:34:00,899 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (DefaultQuartzScheduler_Worker-4) [679f817a] Procceed on vm stop entered: b6972001-2bd6-4cea-9c15-c046674754d4 >2014-01-09 18:34:00,908 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (DefaultQuartzScheduler_Worker-4) [679f817a] Procceed on vm stop entered: 872c3dea-6984-48fa-99c3-354fd367c3f9 >2014-01-09 18:34:00,916 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-4) [3d494ef7] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 18:34:00,922 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-4) [155264a2] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 18:34:00,953 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) [155264a2] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Highly Available VM DC-02 failed. It will be restarted automatically. >2014-01-09 18:34:00,954 INFO [org.ovirt.engine.core.bll.VdsEventListener] (DefaultQuartzScheduler_Worker-4) [155264a2] Highly Available VM went down. Attempting to restart. VM Name: DC-02, VM Id:33f88d50-30eb-45c7-9850-733da17ed7a4 >2014-01-09 18:34:01,094 INFO [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-4) [35ce10cf] Lock Acquired to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 18:34:01,190 WARN [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-4) [35ce10cf] CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,VM_CANNOT_RUN_FROM_CD_WITHOUT_ACTIVE_STORAGE_DOMAIN_ISO >2014-01-09 18:34:01,191 INFO [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-4) [35ce10cf] Lock freed to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 18:34:01,205 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) [35ce10cf] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Restart of the Highly Available VM DC-02 failed. >2014-01-09 18:34:01,214 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) [35ce10cf] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Highly Available VM External_Email failed. It will be restarted automatically. >2014-01-09 18:34:01,215 INFO [org.ovirt.engine.core.bll.VdsEventListener] (DefaultQuartzScheduler_Worker-4) [35ce10cf] Highly Available VM went down. Attempting to restart. VM Name: External_Email, VM Id:872c3dea-6984-48fa-99c3-354fd367c3f9 >2014-01-09 18:34:01,240 INFO [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-4) [4a6e660e] Lock Acquired to object EngineLock [exclusiveLocks= key: 872c3dea-6984-48fa-99c3-354fd367c3f9 value: VM >, sharedLocks= ] >2014-01-09 18:34:01,266 WARN [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-4) [4a6e660e] CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,VM_CANNOT_RUN_FROM_CD_WITHOUT_ACTIVE_STORAGE_DOMAIN_ISO >2014-01-09 18:34:01,267 INFO [org.ovirt.engine.core.bll.RunVmCommand] (DefaultQuartzScheduler_Worker-4) [4a6e660e] Lock freed to object EngineLock [exclusiveLocks= key: 872c3dea-6984-48fa-99c3-354fd367c3f9 value: VM >, sharedLocks= ] >2014-01-09 18:34:01,275 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) [4a6e660e] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Restart of the Highly Available VM External_Email failed. >2014-01-09 18:34:02,646 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-9) spmStart polling ended: taskId = e3412952-bdc7-4979-8389-317ad278acda task status = finished >2014-01-09 18:34:02,820 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-9) spmStart polling ended, spm status: SPM >2014-01-09 18:34:02,825 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-9) START, HSMClearTaskVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, taskId=e3412952-bdc7-4979-8389-317ad278acda), log id: a284468 >2014-01-09 18:34:02,959 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-9) FINISH, HSMClearTaskVDSCommand, log id: a284468 >2014-01-09 18:34:02,960 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-9) FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.businessentities.SpmStatusResult@2648c1fb, log id: 6044a858 >2014-01-09 18:34:02,963 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-9) Initialize Irs proxy from vds: 192.168.100.22 >2014-01-09 18:34:02,969 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host ovirt02 (Address: 192.168.100.22). >2014-01-09 18:34:02,992 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-6-thread-33) START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false), log id: 78dc8d85 >2014-01-09 18:34:03,274 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-6-thread-33) -- executeIrsBrokerCommand: Attempting on storage pool 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d >2014-01-09 18:34:03,278 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (pool-6-thread-33) START, HSMGetAllTasksInfoVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5), log id: 70e7777 >2014-01-09 18:34:03,445 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (pool-6-thread-33) FINISH, HSMGetAllTasksInfoVDSCommand, return: [org.ovirt.engine.core.common.asynctasks.AsyncTaskCreationInfo@3d7d0df8, org.ovirt.engine.core.common.asynctasks.AsyncTaskCreationInfo@5d203451], log id: 70e7777 >2014-01-09 18:34:03,446 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (pool-6-thread-33) FINISH, SPMGetAllTasksInfoVDSCommand, return: [org.ovirt.engine.core.common.asynctasks.AsyncTaskCreationInfo@3d7d0df8, org.ovirt.engine.core.common.asynctasks.AsyncTaskCreationInfo@5d203451], log id: 78dc8d85 >2014-01-09 18:34:03,456 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (pool-6-thread-33) Adding task 763d34b0-8132-461f-914b-e329ef9a052e (Parent Command Unknown, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-01-09 18:34:03,458 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (pool-6-thread-33) Adding task 1f5c4ddb-a230-4107-b13d-5a047bca36e4 (Parent Command Unknown, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-01-09 18:34:03,513 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (pool-6-thread-33) BaseAsyncTask::StartPollingTask: Starting to poll task 763d34b0-8132-461f-914b-e329ef9a052e. >2014-01-09 18:34:03,514 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (pool-6-thread-33) BaseAsyncTask::StartPollingTask: Starting to poll task 1f5c4ddb-a230-4107-b13d-5a047bca36e4. >2014-01-09 18:34:03,514 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (pool-6-thread-33) Discovered 2 tasks on Storage Pool CSEnt, 2 added to manager. >2014-01-09 18:34:08,384 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-19) task id 763d34b0-8132-461f-914b-e329ef9a052e is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-01-09 18:34:08,385 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-19) task id 1f5c4ddb-a230-4107-b13d-5a047bca36e4 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-01-09 18:34:18,387 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-33) task id 763d34b0-8132-461f-914b-e329ef9a052e is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-01-09 18:34:18,388 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-33) task id 1f5c4ddb-a230-4107-b13d-5a047bca36e4 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-01-09 18:34:21,002 ERROR [org.ovirt.engine.core.bll.adbroker.GetRootDSE] (ajp--127.0.0.1-8702-6) Failed to query rootDSE for LDAP server ldap://cytechservices.local:389 due to cytechservices.local:389 >2014-01-09 18:34:21,004 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (ajp--127.0.0.1-8702-6) Failed ldap search server ldap://cytechservices.local:389 using user ryan-a@CYTECHSERVICES.LOCAL due to javax.naming.CommunicationException: cytechservices.local:389 [Root exception is java.net.UnknownHostException: cytechservices.local]. We should try the next server >2014-01-09 18:34:21,005 ERROR [org.ovirt.engine.core.bll.adbroker.LdapAuthenticateUserCommand] (ajp--127.0.0.1-8702-6) Failed authenticating user: ryan-a to domain cytechservices.local. Ldap Query Type is getUserByName >2014-01-09 18:34:21,006 ERROR [org.ovirt.engine.core.bll.LoginUserCommand] (ajp--127.0.0.1-8702-6) USER_FAILED_TO_AUTHENTICATE : ryan-a >2014-01-09 18:34:21,007 WARN [org.ovirt.engine.core.bll.LoginUserCommand] (ajp--127.0.0.1-8702-6) CanDoAction of action LoginUser failed. Reasons:USER_FAILED_TO_AUTHENTICATE >2014-01-09 18:34:21,008 INFO [org.ovirt.engine.api.restapi.security.auth.LoginValidator] (ajp--127.0.0.1-8702-6) Login failure, user: ryan-a domain: cytechservices.local reason: [USER_FAILED_TO_AUTHENTICATE] >2014-01-09 18:34:28,389 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-46) task id 763d34b0-8132-461f-914b-e329ef9a052e is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-01-09 18:34:28,390 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-46) task id 1f5c4ddb-a230-4107-b13d-5a047bca36e4 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-01-09 18:34:38,031 INFO [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-8) Running command: LoginAdminUserCommand internal: false. >2014-01-09 18:34:38,048 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2014-01-09 18:34:38,391 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-59) task id 763d34b0-8132-461f-914b-e329ef9a052e is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-01-09 18:34:38,392 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-59) task id 1f5c4ddb-a230-4107-b13d-5a047bca36e4 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-01-09 18:34:38,461 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp--127.0.0.1-8702-9) Running command: LoginUserCommand internal: false. >2014-01-09 18:34:38,466 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2014-01-09 18:34:48,393 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-75) task id 763d34b0-8132-461f-914b-e329ef9a052e is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-01-09 18:34:48,394 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-75) task id 1f5c4ddb-a230-4107-b13d-5a047bca36e4 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-01-09 18:34:55,202 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-5) [2b16ec74] Lock Acquired to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 18:34:55,249 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-5) [2b16ec74] START, IsVmDuringInitiatingVDSCommand( vmId = b6972001-2bd6-4cea-9c15-c046674754d4), log id: 25b216d3 >2014-01-09 18:34:55,250 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-5) [2b16ec74] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 25b216d3 >2014-01-09 18:34:55,302 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-6-thread-50) [2b16ec74] Running command: RunVmCommand internal: false. Entities affected : ID: b6972001-2bd6-4cea-9c15-c046674754d4 Type: VM >2014-01-09 18:34:55,347 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-6-thread-50) [2b16ec74] START, IsoPrefixVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 5a3ccddd >2014-01-09 18:34:55,388 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-6-thread-50) [2b16ec74] FINISH, IsoPrefixVDSCommand, return: /rhev/data-center/mnt/192.168.101.101:_iso/de2fccd8-63cc-4662-8878-077b4cfb810e/images/11111111-1111-1111-1111-111111111111, log id: 5a3ccddd >2014-01-09 18:34:55,410 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-6-thread-50) [2b16ec74] START, CreateVmVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=b6972001-2bd6-4cea-9c15-c046674754d4, vm=VM [DC-01]), log id: 57d4dac3 >2014-01-09 18:34:55,430 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-50) [2b16ec74] START, CreateVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=b6972001-2bd6-4cea-9c15-c046674754d4, vm=VM [DC-01]), log id: 4c2d24 >2014-01-09 18:34:55,537 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-50) [2b16ec74] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand spiceSslCipherSuite=DEFAULT,memSize=2048,kvmEnable=true,smp=2,vmType=kvm,emulatedMachine=rhel6.4.0,keyboardLayout=en-us,memGuaranteedSize=2048,nice=0,display=qxl,smartcardEnable=false,smpCoresPerSocket=1,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,timeOffset=-3,transparentHugePages=true,vmId=b6972001-2bd6-4cea-9c15-c046674754d4,devices=[Ljava.util.HashMap;@6a84282f,acpiEnable=true,vmName=DC-01,cpuType=Nehalem,custom={device_2b5a7e7d-9e3a-4306-bdfb-1e5a7d8f0388device_d200d0bd-05b3-4026-85a1-e3a14159023ddevice_61468f4d-6cc8-4ca6-aa9e-539dbd106e6cdevice_5935ad87-8eaa-4d20-ae4c-d02cbdc566e6=VmDevice {vmId=b6972001-2bd6-4cea-9c15-c046674754d4, deviceId=5935ad87-8eaa-4d20-ae4c-d02cbdc566e6, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_2b5a7e7d-9e3a-4306-bdfb-1e5a7d8f0388device_d200d0bd-05b3-4026-85a1-e3a14159023d=VmDevice {vmId=b6972001-2bd6-4cea-9c15-c046674754d4, deviceId=d200d0bd-05b3-4026-85a1-e3a14159023d, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_2b5a7e7d-9e3a-4306-bdfb-1e5a7d8f0388device_d200d0bd-05b3-4026-85a1-e3a14159023ddevice_61468f4d-6cc8-4ca6-aa9e-539dbd106e6cdevice_5935ad87-8eaa-4d20-ae4c-d02cbdc566e6device_a7b3aefc-6ed6-4b5c-b19e-7a5f210bca58=VmDevice {vmId=b6972001-2bd6-4cea-9c15-c046674754d4, deviceId=a7b3aefc-6ed6-4b5c-b19e-7a5f210bca58, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}, device_2b5a7e7d-9e3a-4306-bdfb-1e5a7d8f0388=VmDevice {vmId=b6972001-2bd6-4cea-9c15-c046674754d4, deviceId=2b5a7e7d-9e3a-4306-bdfb-1e5a7d8f0388, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_2b5a7e7d-9e3a-4306-bdfb-1e5a7d8f0388device_d200d0bd-05b3-4026-85a1-e3a14159023ddevice_61468f4d-6cc8-4ca6-aa9e-539dbd106e6c=VmDevice {vmId=b6972001-2bd6-4cea-9c15-c046674754d4, deviceId=61468f4d-6cc8-4ca6-aa9e-539dbd106e6c, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}} >2014-01-09 18:34:55,545 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-50) [2b16ec74] FINISH, CreateVDSCommand, log id: 4c2d24 >2014-01-09 18:34:55,552 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-6-thread-50) [2b16ec74] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 57d4dac3 >2014-01-09 18:34:55,553 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-6-thread-50) [2b16ec74] Lock freed to object EngineLock [exclusiveLocks= key: b6972001-2bd6-4cea-9c15-c046674754d4 value: VM >, sharedLocks= ] >2014-01-09 18:34:55,563 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-50) [2b16ec74] Correlation ID: 2b16ec74, Job ID: 9b5ed9b5-8899-48dd-b90a-12b23131c1bc, Call Stack: null, Custom Event ID: -1, Message: VM DC-01 was started by admin@internal (Host: ovirt02). >2014-01-09 18:34:57,505 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-89) VM DC-01 b6972001-2bd6-4cea-9c15-c046674754d4 moved from WaitForLaunch --> PoweringUp >2014-01-09 18:34:57,507 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-89) START, FullListVdsCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02], vmIds=[b6972001-2bd6-4cea-9c15-c046674754d4]), log id: 14b5f88f >2014-01-09 18:34:57,544 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-89) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@36d17ee8, log id: 14b5f88f >2014-01-09 18:34:58,395 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-90) task id 763d34b0-8132-461f-914b-e329ef9a052e is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-01-09 18:34:58,396 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-90) task id 1f5c4ddb-a230-4107-b13d-5a047bca36e4 is in pre-polling period and should not be polled. Pre-polling period is 60,000 millis. >2014-01-09 18:35:08,398 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-3) [3156d8f0] task id 763d34b0-8132-461f-914b-e329ef9a052e has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-01-09 18:35:08,399 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-3) [3156d8f0] task id 763d34b0-8132-461f-914b-e329ef9a052e has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-01-09 18:35:08,400 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-3) [3156d8f0] task id 1f5c4ddb-a230-4107-b13d-5a047bca36e4 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-01-09 18:35:08,400 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (DefaultQuartzScheduler_Worker-3) [3156d8f0] Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-01-09 18:35:08,401 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-3) [3156d8f0] task id 763d34b0-8132-461f-914b-e329ef9a052e has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-01-09 18:35:08,402 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-3) [3156d8f0] task id 1f5c4ddb-a230-4107-b13d-5a047bca36e4 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-01-09 18:35:08,442 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-3) [3156d8f0] task id 763d34b0-8132-461f-914b-e329ef9a052e has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-01-09 18:35:08,444 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-3) [3156d8f0] SPMAsyncTask::PollTask: Polling task 763d34b0-8132-461f-914b-e329ef9a052e (Parent Command Unknown, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'cleanSuccess'. >2014-01-09 18:35:08,445 ERROR [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-3) [3156d8f0] BaseAsyncTask::LogEndTaskFailure: Task 763d34b0-8132-461f-914b-e329ef9a052e (Parent Command Unknown, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended with failure: >-- Result: cleanSuccess >-- Message: VDSGenericException: VDSTaskResultNotSuccessException: TaskState contained successful return code, but a non-success result ('cleanSuccess')., >-- Exception: VDSGenericException: VDSTaskResultNotSuccessException: TaskState contained successful return code, but a non-success result ('cleanSuccess'). >2014-01-09 18:35:08,447 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-3) [3156d8f0] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 763d34b0-8132-461f-914b-e329ef9a052e >2014-01-09 18:35:08,449 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-3) [3156d8f0] START, SPMClearTaskVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, taskId = 763d34b0-8132-461f-914b-e329ef9a052e), log id: 54a8c26f >2014-01-09 18:35:08,452 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-3) [3156d8f0] START, HSMClearTaskVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, taskId=763d34b0-8132-461f-914b-e329ef9a052e), log id: 137530d5 >2014-01-09 18:35:08,491 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-3) [3156d8f0] FINISH, HSMClearTaskVDSCommand, log id: 137530d5 >2014-01-09 18:35:08,492 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-3) [3156d8f0] FINISH, SPMClearTaskVDSCommand, log id: 54a8c26f >2014-01-09 18:35:08,498 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-3) [3156d8f0] BaseAsyncTask::RemoveTaskFromDB: Removed task 763d34b0-8132-461f-914b-e329ef9a052e from DataBase >2014-01-09 18:35:08,499 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-3) [3156d8f0] task id 1f5c4ddb-a230-4107-b13d-5a047bca36e4 has passed pre-polling period time and should be polled. Pre-polling period is 60,000 millis. >2014-01-09 18:35:08,500 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-3) [3156d8f0] SPMAsyncTask::PollTask: Polling task 1f5c4ddb-a230-4107-b13d-5a047bca36e4 (Parent Command Unknown, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'cleanSuccess'. >2014-01-09 18:35:08,501 ERROR [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-3) [3156d8f0] BaseAsyncTask::LogEndTaskFailure: Task 1f5c4ddb-a230-4107-b13d-5a047bca36e4 (Parent Command Unknown, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended with failure: >-- Result: cleanSuccess >-- Message: VDSGenericException: VDSTaskResultNotSuccessException: TaskState contained successful return code, but a non-success result ('cleanSuccess')., >-- Exception: VDSGenericException: VDSTaskResultNotSuccessException: TaskState contained successful return code, but a non-success result ('cleanSuccess'). >2014-01-09 18:35:08,503 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-3) [3156d8f0] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 1f5c4ddb-a230-4107-b13d-5a047bca36e4 >2014-01-09 18:35:08,504 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-3) [3156d8f0] START, SPMClearTaskVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, taskId = 1f5c4ddb-a230-4107-b13d-5a047bca36e4), log id: 11099738 >2014-01-09 18:35:08,507 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-3) [3156d8f0] START, HSMClearTaskVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, taskId=1f5c4ddb-a230-4107-b13d-5a047bca36e4), log id: ca6e645 >2014-01-09 18:35:08,544 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-3) [3156d8f0] FINISH, HSMClearTaskVDSCommand, log id: ca6e645 >2014-01-09 18:35:08,544 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-3) [3156d8f0] FINISH, SPMClearTaskVDSCommand, log id: 11099738 >2014-01-09 18:35:08,546 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (DefaultQuartzScheduler_Worker-3) [3156d8f0] BaseAsyncTask::RemoveTaskFromDB: Removed task 1f5c4ddb-a230-4107-b13d-5a047bca36e4 from DataBase >2014-01-09 18:35:56,513 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-71) VM DC-01 b6972001-2bd6-4cea-9c15-c046674754d4 moved from PoweringUp --> Up >2014-01-09 18:35:56,539 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-71) Correlation ID: 2b16ec74, Job ID: 9b5ed9b5-8899-48dd-b90a-12b23131c1bc, Call Stack: null, Custom Event ID: -1, Message: VM DC-01 started on Host ovirt02 >2014-01-09 18:36:01,958 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp--127.0.0.1-8702-9) [4b1ad610] Lock Acquired to object EngineLock [exclusiveLocks= key: DC-01 value: VM_NAME >, sharedLocks= ] >2014-01-09 18:36:01,979 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp--127.0.0.1-8702-9) [4b1ad610] Running command: UpdateVmCommand internal: false. Entities affected : ID: b6972001-2bd6-4cea-9c15-c046674754d4 Type: VM >2014-01-09 18:36:02,050 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-9) [4b1ad610] Correlation ID: 4b1ad610, Job ID: e964b2b1-0f39-4969-ab23-86b046834e78, Call Stack: null, Custom Event ID: -1, Message: VM DC-01 configuration was updated by admin@internal. >2014-01-09 18:36:02,055 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp--127.0.0.1-8702-9) [4b1ad610] Lock freed to object EngineLock [exclusiveLocks= key: DC-01 value: VM_NAME >, sharedLocks= ] >2014-01-09 18:36:02,475 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (ajp--127.0.0.1-8702-3) MultipleActionsRunner of type AddVmInterface invoked with no actions >2014-01-09 18:36:02,745 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (ajp--127.0.0.1-8702-8) MultipleActionsRunner of type UpdateVmInterface invoked with no actions >2014-01-09 18:36:02,941 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (ajp--127.0.0.1-8702-5) MultipleActionsRunner of type RemoveVmInterface invoked with no actions >2014-01-09 18:36:18,384 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (DefaultQuartzScheduler_Worker-98) Setting new tasks map. The map contains now 0 tasks >2014-01-09 18:36:18,385 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (DefaultQuartzScheduler_Worker-98) Cleared all tasks of pool 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d. >2014-01-09 18:36:18,494 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp--127.0.0.1-8702-10) [103ef369] Lock Acquired to object EngineLock [exclusiveLocks= key: DC-02 value: VM_NAME >, sharedLocks= ] >2014-01-09 18:36:18,511 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp--127.0.0.1-8702-10) [103ef369] Running command: UpdateVmCommand internal: false. Entities affected : ID: 33f88d50-30eb-45c7-9850-733da17ed7a4 Type: VM >2014-01-09 18:36:18,551 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-10) [103ef369] Correlation ID: 103ef369, Job ID: 9247c749-4d0a-4de8-a62a-5c1faefdd7ef, Call Stack: null, Custom Event ID: -1, Message: VM DC-02 configuration was updated by admin@internal. >2014-01-09 18:36:18,557 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp--127.0.0.1-8702-10) [103ef369] Lock freed to object EngineLock [exclusiveLocks= key: DC-02 value: VM_NAME >, sharedLocks= ] >2014-01-09 18:36:19,032 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (ajp--127.0.0.1-8702-1) MultipleActionsRunner of type AddVmInterface invoked with no actions >2014-01-09 18:36:19,225 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (ajp--127.0.0.1-8702-12) MultipleActionsRunner of type UpdateVmInterface invoked with no actions >2014-01-09 18:36:19,412 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (ajp--127.0.0.1-8702-5) MultipleActionsRunner of type RemoveVmInterface invoked with no actions >2014-01-09 18:36:21,484 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-10) [7f457737] Lock Acquired to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 18:36:21,503 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-10) [7f457737] START, IsVmDuringInitiatingVDSCommand( vmId = 33f88d50-30eb-45c7-9850-733da17ed7a4), log id: 5e94dae3 >2014-01-09 18:36:21,504 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-10) [7f457737] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 5e94dae3 >2014-01-09 18:36:21,534 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-6-thread-49) [7f457737] Running command: RunVmCommand internal: false. Entities affected : ID: 33f88d50-30eb-45c7-9850-733da17ed7a4 Type: VM >2014-01-09 18:36:21,570 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-6-thread-49) [7f457737] START, IsoPrefixVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 4a31ee51 >2014-01-09 18:36:21,571 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-6-thread-49) [7f457737] FINISH, IsoPrefixVDSCommand, return: /rhev/data-center/mnt/192.168.101.101:_iso/de2fccd8-63cc-4662-8878-077b4cfb810e/images/11111111-1111-1111-1111-111111111111, log id: 4a31ee51 >2014-01-09 18:36:21,576 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-6-thread-49) [7f457737] START, CreateVmVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, vm=VM [DC-02]), log id: 712452ae >2014-01-09 18:36:21,583 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-49) [7f457737] START, CreateVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, vm=VM [DC-02]), log id: 5c70558b >2014-01-09 18:36:21,723 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-49) [7f457737] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand spiceSslCipherSuite=DEFAULT,memSize=2048,kvmEnable=true,smp=2,vmType=kvm,emulatedMachine=rhel6.4.0,keyboardLayout=en-us,memGuaranteedSize=2048,nice=0,display=qxl,smartcardEnable=false,smpCoresPerSocket=1,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,timeOffset=-18003,transparentHugePages=true,vmId=33f88d50-30eb-45c7-9850-733da17ed7a4,devices=[Ljava.util.HashMap;@54edf7f1,acpiEnable=true,vmName=DC-02,cpuType=Nehalem,custom={device_5e70c85e-03ee-4bd3-b900-7deac0482c5fdevice_63240858-587a-478b-b006-ce505bfd5a6bdevice_6af98674-33e0-4c4f-95f2-4aa9ed1f6079device_7cecbe85-ef9d-488a-9c11-e1e3c815433bdevice_afb15134-f83e-456a-8d3d-30e87b702b4c=VmDevice {vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, deviceId=afb15134-f83e-456a-8d3d-30e87b702b4c, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x04, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}, device_5e70c85e-03ee-4bd3-b900-7deac0482c5fdevice_63240858-587a-478b-b006-ce505bfd5a6bdevice_6af98674-33e0-4c4f-95f2-4aa9ed1f6079=VmDevice {vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, deviceId=6af98674-33e0-4c4f-95f2-4aa9ed1f6079, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_5e70c85e-03ee-4bd3-b900-7deac0482c5fdevice_63240858-587a-478b-b006-ce505bfd5a6bdevice_6af98674-33e0-4c4f-95f2-4aa9ed1f6079device_7cecbe85-ef9d-488a-9c11-e1e3c815433b=VmDevice {vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, deviceId=7cecbe85-ef9d-488a-9c11-e1e3c815433b, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_5e70c85e-03ee-4bd3-b900-7deac0482c5fdevice_63240858-587a-478b-b006-ce505bfd5a6b=VmDevice {vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, deviceId=63240858-587a-478b-b006-ce505bfd5a6b, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_5e70c85e-03ee-4bd3-b900-7deac0482c5f=VmDevice {vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, deviceId=5e70c85e-03ee-4bd3-b900-7deac0482c5f, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}} >2014-01-09 18:36:21,732 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-49) [7f457737] FINISH, CreateVDSCommand, log id: 5c70558b >2014-01-09 18:36:21,737 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-6-thread-49) [7f457737] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 712452ae >2014-01-09 18:36:21,738 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-6-thread-49) [7f457737] Lock freed to object EngineLock [exclusiveLocks= key: 33f88d50-30eb-45c7-9850-733da17ed7a4 value: VM >, sharedLocks= ] >2014-01-09 18:36:21,742 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-49) [7f457737] Correlation ID: 7f457737, Job ID: e26f43fd-4842-49d1-add0-0a268b8c9b7b, Call Stack: null, Custom Event ID: -1, Message: VM DC-02 was started by admin@internal (Host: ovirt01). >2014-01-09 18:36:25,013 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) VM DC-02 33f88d50-30eb-45c7-9850-733da17ed7a4 moved from WaitForLaunch --> PoweringUp >2014-01-09 18:36:25,015 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-12) START, FullListVdsCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01], vmIds=[33f88d50-30eb-45c7-9850-733da17ed7a4]), log id: 74ff9283 >2014-01-09 18:36:25,054 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-12) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@6e9c641c, log id: 74ff9283 >2014-01-09 18:36:28,932 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-10) [7428837a] Lock Acquired to object EngineLock [exclusiveLocks= key: 872c3dea-6984-48fa-99c3-354fd367c3f9 value: VM >, sharedLocks= ] >2014-01-09 18:36:28,956 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-10) [7428837a] START, IsVmDuringInitiatingVDSCommand( vmId = 872c3dea-6984-48fa-99c3-354fd367c3f9), log id: 1eda749b >2014-01-09 18:36:28,957 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-10) [7428837a] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 1eda749b >2014-01-09 18:36:28,984 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-6-thread-50) [7428837a] Running command: RunVmCommand internal: false. Entities affected : ID: 872c3dea-6984-48fa-99c3-354fd367c3f9 Type: VM >2014-01-09 18:36:29,009 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-6-thread-50) [7428837a] START, IsoPrefixVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 16656ee1 >2014-01-09 18:36:29,011 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-6-thread-50) [7428837a] FINISH, IsoPrefixVDSCommand, return: /rhev/data-center/mnt/192.168.101.101:_iso/de2fccd8-63cc-4662-8878-077b4cfb810e/images/11111111-1111-1111-1111-111111111111, log id: 16656ee1 >2014-01-09 18:36:29,017 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-6-thread-50) [7428837a] START, CreateVmVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, vm=VM [External_Email]), log id: 5033c71d >2014-01-09 18:36:29,021 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-50) [7428837a] START, CreateVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, vm=VM [External_Email]), log id: 25376bf4 >2014-01-09 18:36:29,087 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-50) [7428837a] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand spiceSslCipherSuite=DEFAULT,memSize=2048,kvmEnable=true,smp=2,vmType=kvm,emulatedMachine=rhel6.4.0,keyboardLayout=en-us,memGuaranteedSize=2048,pitReinjection=false,nice=0,display=qxl,smartcardEnable=false,smpCoresPerSocket=1,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,timeOffset=-2,transparentHugePages=true,vmId=872c3dea-6984-48fa-99c3-354fd367c3f9,devices=[Ljava.util.HashMap;@3b641077,acpiEnable=true,vmName=External_Email,cpuType=Nehalem,custom={device_f122efcc-e76e-400b-a6f1-60815dcda5fbdevice_916dee6a-d96e-4bad-a237-6ac3581fe13b=VmDevice {vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, deviceId=916dee6a-d96e-4bad-a237-6ac3581fe13b, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x04, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}, device_f122efcc-e76e-400b-a6f1-60815dcda5fbdevice_916dee6a-d96e-4bad-a237-6ac3581fe13bdevice_b0a2c9ac-32d7-416c-bbe4-84f5e049353c=VmDevice {vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, deviceId=b0a2c9ac-32d7-416c-bbe4-84f5e049353c, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_f122efcc-e76e-400b-a6f1-60815dcda5fb=VmDevice {vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, deviceId=f122efcc-e76e-400b-a6f1-60815dcda5fb, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_f122efcc-e76e-400b-a6f1-60815dcda5fbdevice_916dee6a-d96e-4bad-a237-6ac3581fe13bdevice_b0a2c9ac-32d7-416c-bbe4-84f5e049353cdevice_f9edcb08-fb7e-4a4d-b681-4dec365e8cc2device_763b89c0-1079-4876-9e96-1a3235b38a1a=VmDevice {vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, deviceId=763b89c0-1079-4876-9e96-1a3235b38a1a, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_f122efcc-e76e-400b-a6f1-60815dcda5fbdevice_916dee6a-d96e-4bad-a237-6ac3581fe13bdevice_b0a2c9ac-32d7-416c-bbe4-84f5e049353cdevice_f9edcb08-fb7e-4a4d-b681-4dec365e8cc2=VmDevice {vmId=872c3dea-6984-48fa-99c3-354fd367c3f9, deviceId=f9edcb08-fb7e-4a4d-b681-4dec365e8cc2, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}} >2014-01-09 18:36:29,097 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-50) [7428837a] FINISH, CreateVDSCommand, log id: 25376bf4 >2014-01-09 18:36:29,101 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-6-thread-50) [7428837a] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 5033c71d >2014-01-09 18:36:29,102 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-6-thread-50) [7428837a] Lock freed to object EngineLock [exclusiveLocks= key: 872c3dea-6984-48fa-99c3-354fd367c3f9 value: VM >, sharedLocks= ] >2014-01-09 18:36:29,106 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-50) [7428837a] Correlation ID: 7428837a, Job ID: 6fe0d3e4-3f92-4d1d-b739-a814ff1e7b94, Call Stack: null, Custom Event ID: -1, Message: VM External_Email was started by admin@internal (Host: ovirt01). >2014-01-09 18:36:31,334 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-17) VM External_Email 872c3dea-6984-48fa-99c3-354fd367c3f9 moved from WaitForLaunch --> PoweringUp >2014-01-09 18:36:31,336 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-17) START, FullListVdsCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01], vmIds=[872c3dea-6984-48fa-99c3-354fd367c3f9]), log id: 3bd339a0 >2014-01-09 18:36:31,381 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-17) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@60860784, log id: 3bd339a0 >2014-01-09 18:37:24,476 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) VM DC-02 33f88d50-30eb-45c7-9850-733da17ed7a4 moved from PoweringUp --> Up >2014-01-09 18:37:24,484 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-88) Correlation ID: 7f457737, Job ID: e26f43fd-4842-49d1-add0-0a268b8c9b7b, Call Stack: null, Custom Event ID: -1, Message: VM DC-02 started on Host ovirt01 >2014-01-09 18:37:30,659 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-98) VM External_Email 872c3dea-6984-48fa-99c3-354fd367c3f9 moved from PoweringUp --> Up >2014-01-09 18:37:30,666 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-98) Correlation ID: 7428837a, Job ID: 6fe0d3e4-3f92-4d1d-b739-a814ff1e7b94, Call Stack: null, Custom Event ID: -1, Message: VM External_Email started on Host ovirt01 >2014-01-09 18:38:48,190 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetIsoListVDSCommand] (pool-6-thread-50) START, GetIsoListVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false), log id: 27e2907f >2014-01-09 18:38:48,193 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HsmGetIsoListVDSCommand] (pool-6-thread-50) START, HsmGetIsoListVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 7d86f935 >2014-01-09 18:38:48,270 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HsmGetIsoListVDSCommand] (pool-6-thread-50) FINISH, HsmGetIsoListVDSCommand, return: [virtio-win-0.1-52.iso, centos64x64netinst.iso, virtio-win-0.1-65.iso, winserve2012.iso, kali-linux-1.0.5-amd641.iso, Server2008R2SP1-en-US-July2013.iso, win7.iso, virt-p2v-0.9.1-2.20130730.1.el6.centos.iso, zentyal-3.2-amd64.iso, Elastix-2.4.0-Stable-x86_64-bin-04feb2013.iso, EXCH201064.ISO], log id: 7d86f935 >2014-01-09 18:38:48,272 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetIsoListVDSCommand] (pool-6-thread-50) FINISH, GetIsoListVDSCommand, return: [virtio-win-0.1-52.iso, centos64x64netinst.iso, virtio-win-0.1-65.iso, winserve2012.iso, kali-linux-1.0.5-amd641.iso, Server2008R2SP1-en-US-July2013.iso, win7.iso, virt-p2v-0.9.1-2.20130730.1.el6.centos.iso, zentyal-3.2-amd64.iso, Elastix-2.4.0-Stable-x86_64-bin-04feb2013.iso, EXCH201064.ISO], log id: 27e2907f >2014-01-09 18:38:48,292 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (pool-6-thread-50) Finished automatic refresh process for ISO file type with success, for storage domain id de2fccd8-63cc-4662-8878-077b4cfb810e. >2014-01-09 18:52:55,544 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetStatsVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 18:52:55,545 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) vds::refreshVdsStats Failed getVdsStats, vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, error = VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 18:52:55,549 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-75) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 18:53:00,557 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-81) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 18:53:00,559 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-81) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 18:53:05,570 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 18:53:05,572 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-85) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 18:53:10,579 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-93) Command GetCapabilitiesVDS execution failed. Exception: VDSNetworkException: java.net.SocketTimeoutException: connect timed out >2014-01-09 18:53:10,581 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-93) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.SocketTimeoutException: connect timed out >2014-01-09 18:53:13,713 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-96) START, GetHardwareInfoVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01]), log id: 517af8f2 >2014-01-09 18:53:13,757 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-96) FINISH, GetHardwareInfoVDSCommand, log id: 517af8f2 >2014-01-09 18:53:13,770 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-96) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host ovirt01. >2014-01-09 18:53:13,805 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-96) [7d1235b8] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 18:53:13,808 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-96) [3a3ac0e] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 18:53:13,861 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-96) [3a3ac0e] START, ConnectStorageServerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = ISCSI, connectionList = [{ id: 623aa506-12d1-4437-8e81-f39b2317f3dd, connection: 192.168.101.101, iqn: iqn.2013-11.local.cytechservices.n16000pro:iscsi.hosts-tgt00.raid0, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 4fb74395 >2014-01-09 18:53:16,523 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-96) [3a3ac0e] FINISH, ConnectStorageServerVDSCommand, return: {623aa506-12d1-4437-8e81-f39b2317f3dd=0}, log id: 4fb74395 >2014-01-09 18:53:16,524 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-96) [3a3ac0e] Host ovirt01 storage connection was succeeded >2014-01-09 18:53:16,526 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-96) [3a3ac0e] START, ConnectStorageServerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 210997b7-9098-48d6-8e97-9689085aa4b0, connection: 192.168.101.101:/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: AUTO, nfsRetrans: null, nfsTimeo: null };]), log id: 4f27a50e >2014-01-09 18:53:16,570 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-96) [3a3ac0e] FINISH, ConnectStorageServerVDSCommand, return: {210997b7-9098-48d6-8e97-9689085aa4b0=0}, log id: 4f27a50e >2014-01-09 18:53:16,570 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-96) [3a3ac0e] Host ovirt01 storage connection was succeeded >2014-01-09 18:53:16,572 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-96) [3a3ac0e] START, ConnectStorageServerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 4e5d4030-1bd2-4487-b08f-2135eba3f11f, connection: 192.168.101.101:/vms, iqn: null, vfsType: null, mountOptions: null, nfsVersion: V4, nfsRetrans: null, nfsTimeo: null };]), log id: 5800536e >2014-01-09 18:53:16,613 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-96) [3a3ac0e] FINISH, ConnectStorageServerVDSCommand, return: {4e5d4030-1bd2-4487-b08f-2135eba3f11f=0}, log id: 5800536e >2014-01-09 18:53:16,613 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-96) [3a3ac0e] Host ovirt01 storage connection was succeeded >2014-01-09 18:53:16,618 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-6-thread-50) START, ConnectStoragePoolVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, vds_spm_id = 1, masterDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, masterVersion = 2), log id: 651b9b29 >2014-01-09 18:53:16,657 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-6-thread-50) FINISH, ConnectStoragePoolVDSCommand, log id: 651b9b29 >2014-01-09 18:53:16,701 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-96) [3a3ac0e] START, SetMOMPolicyParametersVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8), log id: 390ad5da >2014-01-09 18:53:16,803 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-96) [3a3ac0e] FINISH, SetMOMPolicyParametersVDSCommand, log id: 390ad5da >2014-01-09 18:53:16,808 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-96) [3a3ac0e] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host ovirt01. >2014-01-09 18:53:16,813 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-96) [6008867f] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 18:53:16,817 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-96) [60cb7ef6] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 18:53:17,007 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-96) [60cb7ef6] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage domain thecus-svcs-lun05 experienced a high latency of 18.8255 seconds from host ovirt01. This may cause performance and functional issues. Please consult your Storage Administrator. >2014-01-09 18:53:49,842 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (ajp--127.0.0.1-8702-2) [318a9814] Lock Acquired to object EngineLock [exclusiveLocks= key: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 value: VDS >, sharedLocks= ] >2014-01-09 18:53:49,845 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (pool-6-thread-49) [318a9814] Running command: RefreshHostCapabilitiesCommand internal: false. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 18:53:49,847 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (pool-6-thread-49) [318a9814] Before acquiring lock in order to prevent monitoring for host ovirt01 from data-center CSEnt >2014-01-09 18:53:49,848 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (pool-6-thread-49) [318a9814] Lock acquired, from now a monitoring of host will be skipped for host ovirt01 from data-center CSEnt >2014-01-09 18:53:49,966 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (pool-6-thread-49) [318a9814] START, GetHardwareInfoVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01]), log id: 52c57b3d >2014-01-09 18:53:50,005 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (pool-6-thread-49) [318a9814] FINISH, GetHardwareInfoVDSCommand, log id: 52c57b3d >2014-01-09 18:53:50,018 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (pool-6-thread-49) [4640431] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 18:53:50,022 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (pool-6-thread-49) [d80feb1] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 18:53:50,024 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (pool-6-thread-49) [d80feb1] Refresh host capabilities finished. Lock released. Monitoring can run now for host ovirt01 from data-center CSEnt >2014-01-09 18:53:50,029 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-49) [d80feb1] Correlation ID: 318a9814, Call Stack: null, Custom Event ID: -1, Message: Successfully refreshed the capabilities of host ovirt01. >2014-01-09 18:53:50,030 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (pool-6-thread-49) [d80feb1] Lock freed to object EngineLock [exclusiveLocks= key: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 value: VDS >, sharedLocks= ] >2014-01-09 18:54:14,377 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp--127.0.0.1-8702-12) [65b7f821] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 33f88d50-30eb-45c7-9850-733da17ed7a4 Type: VM >2014-01-09 18:54:14,396 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp--127.0.0.1-8702-12) [65b7f821] START, SetVmTicketVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vmId=33f88d50-30eb-45c7-9850-733da17ed7a4, ticket=SDqcXcjZiMKx, validTime=120,m userName=admin@internal, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 6d8cfae2 >2014-01-09 18:54:14,464 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp--127.0.0.1-8702-12) [65b7f821] FINISH, SetVmTicketVDSCommand, log id: 6d8cfae2 >2014-01-09 18:54:14,473 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-12) [65b7f821] Correlation ID: 65b7f821, Call Stack: null, Custom Event ID: -1, Message: user admin@internal initiated console session for VM DC-02 >2014-01-09 18:54:39,044 ERROR [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-9) USER_FAILED_TO_AUTHENTICATE : patrick-a >2014-01-09 18:54:39,044 WARN [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-9) CanDoAction of action LoginAdminUser failed. Reasons:USER_FAILED_TO_AUTHENTICATE >2014-01-09 18:54:51,539 ERROR [org.ovirt.engine.core.bll.adbroker.GetRootDSE] (ajp--127.0.0.1-8702-10) Failed to query rootDSE for LDAP server ldap://cytechservices.local:389 due to cytechservices.local:389 >2014-01-09 18:54:51,540 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (ajp--127.0.0.1-8702-10) Failed ldap search server ldap://cytechservices.local:389 using user patrick-a@CYTECHSERVICES.LOCAL due to javax.naming.CommunicationException: cytechservices.local:389 [Root exception is java.net.UnknownHostException: cytechservices.local]. We should try the next server >2014-01-09 18:54:51,542 ERROR [org.ovirt.engine.core.bll.adbroker.LdapAuthenticateUserCommand] (ajp--127.0.0.1-8702-10) Failed authenticating user: patrick-a to domain cytechservices.local. Ldap Query Type is getUserByName >2014-01-09 18:54:51,542 ERROR [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-10) USER_FAILED_TO_AUTHENTICATE : patrick-a >2014-01-09 18:54:51,544 WARN [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-10) CanDoAction of action LoginAdminUser failed. Reasons:USER_FAILED_TO_AUTHENTICATE >2014-01-09 18:54:58,902 ERROR [org.ovirt.engine.core.bll.adbroker.GetRootDSE] (ajp--127.0.0.1-8702-2) Failed to query rootDSE for LDAP server ldap://cytechservices.local:389 due to cytechservices.local:389 >2014-01-09 18:54:58,904 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (ajp--127.0.0.1-8702-2) Failed ldap search server ldap://cytechservices.local:389 using user patrick-a@CYTECHSERVICES.LOCAL due to javax.naming.CommunicationException: cytechservices.local:389 [Root exception is java.net.UnknownHostException: cytechservices.local]. We should try the next server >2014-01-09 18:54:58,905 ERROR [org.ovirt.engine.core.bll.adbroker.LdapAuthenticateUserCommand] (ajp--127.0.0.1-8702-2) Failed authenticating user: patrick-a to domain cytechservices.local. Ldap Query Type is getUserByName >2014-01-09 18:54:58,906 ERROR [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-2) USER_FAILED_TO_AUTHENTICATE : patrick-a >2014-01-09 18:54:58,906 WARN [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-2) CanDoAction of action LoginAdminUser failed. Reasons:USER_FAILED_TO_AUTHENTICATE >2014-01-09 18:55:08,905 ERROR [org.ovirt.engine.core.bll.adbroker.GetRootDSE] (ajp--127.0.0.1-8702-5) Failed to query rootDSE for LDAP server ldap://cytechservices.local:389 due to cytechservices.local:389 >2014-01-09 18:55:08,907 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (ajp--127.0.0.1-8702-5) Failed ldap search server ldap://cytechservices.local:389 using user patrick.denton@CYTECHSERVICES.LOCAL due to javax.naming.CommunicationException: cytechservices.local:389 [Root exception is java.net.UnknownHostException: cytechservices.local]. We should try the next server >2014-01-09 18:55:08,908 ERROR [org.ovirt.engine.core.bll.adbroker.LdapAuthenticateUserCommand] (ajp--127.0.0.1-8702-5) Failed authenticating user: patrick.denton to domain cytechservices.local. Ldap Query Type is getUserByName >2014-01-09 18:55:08,909 ERROR [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-5) USER_FAILED_TO_AUTHENTICATE : patrick.denton >2014-01-09 18:55:08,909 WARN [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-5) CanDoAction of action LoginAdminUser failed. Reasons:USER_FAILED_TO_AUTHENTICATE >2014-01-09 18:55:39,097 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp--127.0.0.1-8702-9) [2328be8] Running command: LogoutUserCommand internal: false. >2014-01-09 18:55:39,102 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-9) [2328be8] Correlation ID: 2328be8, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2014-01-09 18:55:46,880 ERROR [org.ovirt.engine.core.bll.adbroker.GetRootDSE] (ajp--127.0.0.1-8702-5) Failed to query rootDSE for LDAP server ldap://cytechservices.local:389 due to cytechservices.local:389 >2014-01-09 18:55:46,882 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (ajp--127.0.0.1-8702-5) Failed ldap search server ldap://cytechservices.local:389 using user ryan-a@CYTECHSERVICES.LOCAL due to javax.naming.CommunicationException: cytechservices.local:389 [Root exception is java.net.UnknownHostException: cytechservices.local]. We should try the next server >2014-01-09 18:55:46,883 ERROR [org.ovirt.engine.core.bll.adbroker.LdapAuthenticateUserCommand] (ajp--127.0.0.1-8702-5) Failed authenticating user: ryan-a to domain cytechservices.local. Ldap Query Type is getUserByName >2014-01-09 18:55:46,883 ERROR [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-5) USER_FAILED_TO_AUTHENTICATE : ryan-a >2014-01-09 18:55:46,884 WARN [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-5) CanDoAction of action LoginAdminUser failed. Reasons:USER_FAILED_TO_AUTHENTICATE >2014-01-09 18:56:43,513 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand] (DefaultQuartzScheduler_Worker-89) Command ListVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Network is unreachable >2014-01-09 18:56:43,516 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-89) Failed to refresh VDS , vds = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 : ovirt02, VDS Network Error, continuing. >java.net.ConnectException: Network is unreachable >2014-01-09 18:56:44,814 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVDSCommand] (DefaultQuartzScheduler_Worker-85) Command ListVDS execution failed. Exception: VDSNetworkException: java.net.ConnectException: Network is unreachable >2014-01-09 18:56:44,818 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-85) Failed to refresh VDS , vds = f2d7a8a9-e77e-4e9a-8031-23a0731407c8 : ovirt01, VDS Network Error, continuing. >java.net.ConnectException: Network is unreachable >2014-01-09 18:56:47,673 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-88) START, GetHardwareInfoVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02]), log id: 4f6dec03 >2014-01-09 18:56:47,723 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-88) FINISH, GetHardwareInfoVDSCommand, log id: 4f6dec03 >2014-01-09 18:56:47,744 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-88) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host ovirt02. >2014-01-09 18:56:47,778 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-88) [b7aa95b] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 18:56:47,782 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-88) [4bda2f6b] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 18:56:47,828 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-88) [4bda2f6b] START, ConnectStorageServerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = ISCSI, connectionList = [{ id: 623aa506-12d1-4437-8e81-f39b2317f3dd, connection: 192.168.101.101, iqn: iqn.2013-11.local.cytechservices.n16000pro:iscsi.hosts-tgt00.raid0, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 270ec08f >2014-01-09 18:56:47,938 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-94) START, GetHardwareInfoVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, vds=Host[ovirt01]), log id: 6d5bc7b7 >2014-01-09 18:56:47,973 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-94) FINISH, GetHardwareInfoVDSCommand, log id: 6d5bc7b7 >2014-01-09 18:56:47,985 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-94) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: State was set to Up for host ovirt01. >2014-01-09 18:56:48,021 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-94) [4886d461] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 18:56:48,023 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-94) [6adf7800] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d Type: StoragePool >2014-01-09 18:56:48,069 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-94) [6adf7800] START, ConnectStorageServerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = ISCSI, connectionList = [{ id: 623aa506-12d1-4437-8e81-f39b2317f3dd, connection: 192.168.101.101, iqn: iqn.2013-11.local.cytechservices.n16000pro:iscsi.hosts-tgt00.raid0, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 77219647 >2014-01-09 18:56:48,149 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-88) [4bda2f6b] FINISH, ConnectStorageServerVDSCommand, return: {623aa506-12d1-4437-8e81-f39b2317f3dd=0}, log id: 270ec08f >2014-01-09 18:56:48,150 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-88) [4bda2f6b] Host ovirt02 storage connection was succeeded >2014-01-09 18:56:48,153 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-88) [4bda2f6b] START, ConnectStorageServerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 210997b7-9098-48d6-8e97-9689085aa4b0, connection: 192.168.101.101:/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: AUTO, nfsRetrans: null, nfsTimeo: null };]), log id: 46bad0b9 >2014-01-09 18:56:48,197 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-88) [4bda2f6b] FINISH, ConnectStorageServerVDSCommand, return: {210997b7-9098-48d6-8e97-9689085aa4b0=0}, log id: 46bad0b9 >2014-01-09 18:56:48,198 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-88) [4bda2f6b] Host ovirt02 storage connection was succeeded >2014-01-09 18:56:48,200 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-88) [4bda2f6b] START, ConnectStorageServerVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 4e5d4030-1bd2-4487-b08f-2135eba3f11f, connection: 192.168.101.101:/vms, iqn: null, vfsType: null, mountOptions: null, nfsVersion: V4, nfsRetrans: null, nfsTimeo: null };]), log id: 4a32ac89 >2014-01-09 18:56:48,242 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-88) [4bda2f6b] FINISH, ConnectStorageServerVDSCommand, return: {4e5d4030-1bd2-4487-b08f-2135eba3f11f=0}, log id: 4a32ac89 >2014-01-09 18:56:48,243 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-88) [4bda2f6b] Host ovirt02 storage connection was succeeded >2014-01-09 18:56:48,248 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-6-thread-50) START, ConnectStoragePoolVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, vds_spm_id = 2, masterDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, masterVersion = 2), log id: 4944e75f >2014-01-09 18:56:48,287 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-6-thread-50) FINISH, ConnectStoragePoolVDSCommand, log id: 4944e75f >2014-01-09 18:56:48,330 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-88) [4bda2f6b] START, SetMOMPolicyParametersVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5), log id: 42dc30c >2014-01-09 18:56:48,408 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-94) [6adf7800] FINISH, ConnectStorageServerVDSCommand, return: {623aa506-12d1-4437-8e81-f39b2317f3dd=0}, log id: 77219647 >2014-01-09 18:56:48,409 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-94) [6adf7800] Host ovirt01 storage connection was succeeded >2014-01-09 18:56:48,411 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-94) [6adf7800] START, ConnectStorageServerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 210997b7-9098-48d6-8e97-9689085aa4b0, connection: 192.168.101.101:/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: AUTO, nfsRetrans: null, nfsTimeo: null };]), log id: 19da0d1c >2014-01-09 18:56:48,434 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-88) [4bda2f6b] FINISH, SetMOMPolicyParametersVDSCommand, log id: 42dc30c >2014-01-09 18:56:48,438 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-88) [4bda2f6b] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host ovirt02. >2014-01-09 18:56:48,444 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-88) [77814146] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 18:56:48,451 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-88) [77412688] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5 Type: VDS >2014-01-09 18:56:48,453 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-94) [6adf7800] FINISH, ConnectStorageServerVDSCommand, return: {210997b7-9098-48d6-8e97-9689085aa4b0=0}, log id: 19da0d1c >2014-01-09 18:56:48,454 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-94) [6adf7800] Host ovirt01 storage connection was succeeded >2014-01-09 18:56:48,455 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-94) [6adf7800] START, ConnectStorageServerVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, storageType = NFS, connectionList = [{ id: 4e5d4030-1bd2-4487-b08f-2135eba3f11f, connection: 192.168.101.101:/vms, iqn: null, vfsType: null, mountOptions: null, nfsVersion: V4, nfsRetrans: null, nfsTimeo: null };]), log id: 690eb47f >2014-01-09 18:56:48,496 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-94) [6adf7800] FINISH, ConnectStorageServerVDSCommand, return: {4e5d4030-1bd2-4487-b08f-2135eba3f11f=0}, log id: 690eb47f >2014-01-09 18:56:48,497 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-94) [6adf7800] Host ovirt01 storage connection was succeeded >2014-01-09 18:56:48,502 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-6-thread-49) START, ConnectStoragePoolVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8, storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, vds_spm_id = 1, masterDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, masterVersion = 2), log id: 47c32473 >2014-01-09 18:56:48,540 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (pool-6-thread-49) FINISH, ConnectStoragePoolVDSCommand, log id: 47c32473 >2014-01-09 18:56:48,582 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-94) [6adf7800] START, SetMOMPolicyParametersVDSCommand(HostName = ovirt01, HostId = f2d7a8a9-e77e-4e9a-8031-23a0731407c8), log id: 31f80b4d >2014-01-09 18:56:48,684 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-94) [6adf7800] FINISH, SetMOMPolicyParametersVDSCommand, log id: 31f80b4d >2014-01-09 18:56:48,688 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-94) [6adf7800] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host ovirt01. >2014-01-09 18:56:48,693 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-94) [428420bf] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 18:56:48,697 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-94) [a81a815] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: f2d7a8a9-e77e-4e9a-8031-23a0731407c8 Type: VDS >2014-01-09 18:56:56,255 ERROR [org.ovirt.engine.core.bll.adbroker.GetRootDSE] (ajp--127.0.0.1-8702-7) Failed to query rootDSE for LDAP server ldap://cytechservices.local:389 due to cytechservices.local:389 >2014-01-09 18:56:56,257 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (ajp--127.0.0.1-8702-7) Failed ldap search server ldap://cytechservices.local:389 using user patrick.denton@CYTECHSERVICES.LOCAL due to javax.naming.CommunicationException: cytechservices.local:389 [Root exception is java.net.UnknownHostException: cytechservices.local]. We should try the next server >2014-01-09 18:56:56,258 ERROR [org.ovirt.engine.core.bll.adbroker.LdapAuthenticateUserCommand] (ajp--127.0.0.1-8702-7) Failed authenticating user: patrick.denton to domain cytechservices.local. Ldap Query Type is getUserByName >2014-01-09 18:56:56,259 ERROR [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-7) USER_FAILED_TO_AUTHENTICATE : patrick.denton >2014-01-09 18:56:56,260 WARN [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-7) CanDoAction of action LoginAdminUser failed. Reasons:USER_FAILED_TO_AUTHENTICATE >2014-01-09 18:57:01,648 ERROR [org.ovirt.engine.core.bll.adbroker.GetRootDSE] (ajp--127.0.0.1-8702-8) Failed to query rootDSE for LDAP server ldap://cytechservices.local:389 due to cytechservices.local:389 >2014-01-09 18:57:01,650 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (ajp--127.0.0.1-8702-8) Failed ldap search server ldap://cytechservices.local:389 using user ryan-a@CYTECHSERVICES.LOCAL due to javax.naming.CommunicationException: cytechservices.local:389 [Root exception is java.net.UnknownHostException: cytechservices.local]. We should try the next server >2014-01-09 18:57:01,651 ERROR [org.ovirt.engine.core.bll.adbroker.LdapAuthenticateUserCommand] (ajp--127.0.0.1-8702-8) Failed authenticating user: ryan-a to domain cytechservices.local. Ldap Query Type is getUserByName >2014-01-09 18:57:01,652 ERROR [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-8) USER_FAILED_TO_AUTHENTICATE : ryan-a >2014-01-09 18:57:01,653 WARN [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-8) CanDoAction of action LoginAdminUser failed. Reasons:USER_FAILED_TO_AUTHENTICATE >2014-01-09 19:14:49,305 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-90) Failed to invoke scheduled method gluster_async_task_poll_event: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor112.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:52) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:14:49,388 ERROR [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (DefaultQuartzScheduler_Worker-97) Error updating DWH Heart Beat: : org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1001) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.call(JdbcTemplate.java:1064) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.AbstractJdbcCall.executeCallInternal(AbstractJdbcCall.java:388) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.AbstractJdbcCall.doExecute(AbstractJdbcCall.java:351) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeModification(SimpleJdbcCallsHandler.java:74) [dal.jar:] > at org.ovirt.engine.core.dao.dwh.DwhHistoryTimekeepingDaoDbFacadeImpl.save(DwhHistoryTimekeepingDaoDbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.bll.dwh.DwhHeartBeat.engineIsRunningNotification(DwhHeartBeat.java:42) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor109.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 15 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 17 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 20 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 23 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 26 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 35 more > >2014-01-09 19:14:49,442 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-88) Failed to invoke scheduled method PerformLoadBalancing: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor113.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.scheduling.SchedulingManager.PerformLoadBalancing(SchedulingManager.java:678) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:14:50,188 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-98) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor77.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:14:50,209 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-6) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:15:01,204 INFO [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-12) Running command: LoginAdminUserCommand internal: false. >2014-01-09 19:15:01,219 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2014-01-09 19:19:35,252 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-91) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:19:35,279 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-86) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:19:35,342 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-89) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor77.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:19:35,357 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-85) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:19:37,955 ERROR [org.ovirt.engine.core.bll.SearchQuery] (ajp--127.0.0.1-8702-6) Query SearchQuery failed. Exception message is Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource >2014-01-09 19:19:38,287 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-87) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:19:38,311 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-92) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:19:40,384 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-90) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor77.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:19:40,411 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-95) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:19:41,325 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-97) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:19:41,345 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-88) [77412688] Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:19:42,948 ERROR [org.ovirt.engine.core.bll.SearchQuery] (ajp--127.0.0.1-8702-2) Query SearchQuery failed. Exception message is Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource >2014-01-09 19:19:44,364 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:19:44,391 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-94) [a81a815] Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:19:45,435 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-100) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor77.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:19:45,463 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-99) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:19:47,410 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-96) [60cb7ef6] Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:19:47,436 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-2) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:19:47,962 ERROR [org.ovirt.engine.core.bll.SearchQuery] (ajp--127.0.0.1-8702-7) Query SearchQuery failed. Exception message is Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource >2014-01-09 19:19:49,348 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-1) Failed to invoke scheduled method gluster_async_task_poll_event: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor112.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:52) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:19:49,437 ERROR [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (DefaultQuartzScheduler_Worker-10) Error updating DWH Heart Beat: : org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1001) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.call(JdbcTemplate.java:1064) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.AbstractJdbcCall.executeCallInternal(AbstractJdbcCall.java:388) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.AbstractJdbcCall.doExecute(AbstractJdbcCall.java:351) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeModification(SimpleJdbcCallsHandler.java:74) [dal.jar:] > at org.ovirt.engine.core.dao.dwh.DwhHistoryTimekeepingDaoDbFacadeImpl.save(DwhHistoryTimekeepingDaoDbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.bll.dwh.DwhHeartBeat.engineIsRunningNotification(DwhHeartBeat.java:42) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor109.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 15 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 17 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 20 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 23 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 26 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 35 more > >2014-01-09 19:19:49,534 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-8) Failed to invoke scheduled method PerformLoadBalancing: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor113.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.scheduling.SchedulingManager.PerformLoadBalancing(SchedulingManager.java:678) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:19:50,453 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-11) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:19:50,478 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-3) [3156d8f0] Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:19:50,499 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-16) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor77.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:19:50,523 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-13) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:19:52,964 ERROR [org.ovirt.engine.core.bll.SearchQuery] (ajp--127.0.0.1-8702-12) Query SearchQuery failed. Exception message is Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource >2014-01-09 19:19:53,503 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-15) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:19:53,529 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-14) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:19:55,551 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-19) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor77.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:19:55,578 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-9) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:19:56,543 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-17) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:19:56,572 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:19:57,969 ERROR [org.ovirt.engine.core.bll.SearchQuery] (ajp--127.0.0.1-8702-5) Query SearchQuery failed. Exception message is Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource >2014-01-09 19:19:59,140 ERROR [org.ovirt.engine.core.bll.GetJobsByOffsetQuery] (ajp--127.0.0.1-8702-11) Query GetJobsByOffsetQuery failed. Exception message is Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource >2014-01-09 19:19:59,222 ERROR [org.ovirt.engine.core.bll.SearchQuery] (ajp--127.0.0.1-8702-13) Query SearchQuery failed. Exception message is Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource >2014-01-09 19:19:59,318 ERROR [org.ovirt.engine.core.bll.SearchQuery] (ajp--127.0.0.1-8702-1) Query SearchQuery failed. Exception message is Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource >2014-01-09 19:19:59,586 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-22) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:19:59,605 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-20) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:20:00,021 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-26) Failed to invoke scheduled method onTimer: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_45] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.listFailedAutorecoverables(VdsDAODbFacadeImpl.java:188) [dal.jar:] > at org.ovirt.engine.core.bll.AutoRecoveryManager.check(AutoRecoveryManager.java:121) [bll.jar:] > at org.ovirt.engine.core.bll.AutoRecoveryManager.onTimer(AutoRecoveryManager.java:59) [bll.jar:] > ... 7 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:20:00,605 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-25) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor77.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:20:00,620 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-23) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:20:02,625 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-31) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:20:02,642 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-24) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:20:02,957 ERROR [org.ovirt.engine.core.bll.SearchQuery] (ajp--127.0.0.1-8702-3) Query SearchQuery failed. Exception message is Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource >2014-01-09 19:20:05,646 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-27) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor77.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:20:05,677 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-30) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:20:05,700 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-34) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:20:05,727 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-32) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:20:07,984 ERROR [org.ovirt.engine.core.bll.SearchQuery] (ajp--127.0.0.1-8702-9) Query SearchQuery failed. Exception message is Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource >2014-01-09 19:20:08,764 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-35) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:20:08,791 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-36) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:20:10,695 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-33) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor77.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:20:10,752 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-40) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:20:11,813 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-37) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:20:11,843 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-39) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:20:13,003 ERROR [org.ovirt.engine.core.bll.SearchQuery] (ajp--127.0.0.1-8702-8) Query SearchQuery failed. Exception message is Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource >2014-01-09 19:20:14,864 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:20:14,879 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:20:15,735 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-41) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor77.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:20:15,789 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-45) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:20:17,904 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-48) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:20:17,919 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-46) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:20:17,966 ERROR [org.ovirt.engine.core.bll.SearchQuery] (ajp--127.0.0.1-8702-4) Query SearchQuery failed. Exception message is Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource >2014-01-09 19:20:19,469 ERROR [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (DefaultQuartzScheduler_Worker-47) Error updating DWH Heart Beat: : org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1001) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.call(JdbcTemplate.java:1064) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.AbstractJdbcCall.executeCallInternal(AbstractJdbcCall.java:388) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.AbstractJdbcCall.doExecute(AbstractJdbcCall.java:351) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeModification(SimpleJdbcCallsHandler.java:74) [dal.jar:] > at org.ovirt.engine.core.dao.dwh.DwhHistoryTimekeepingDaoDbFacadeImpl.save(DwhHistoryTimekeepingDaoDbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.bll.dwh.DwhHeartBeat.engineIsRunningNotification(DwhHeartBeat.java:42) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor109.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 15 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 17 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 20 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 23 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 26 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 35 more > >2014-01-09 19:20:20,775 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-51) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor77.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:20:20,825 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-52) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:20:20,944 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-53) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:20:20,971 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-55) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:20:22,982 ERROR [org.ovirt.engine.core.bll.SearchQuery] (ajp--127.0.0.1-8702-6) Query SearchQuery failed. Exception message is Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource >2014-01-09 19:20:23,988 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-54) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:20:24,015 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-56) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:20:25,811 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-60) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor77.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:20:25,862 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-57) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:20:27,036 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-58) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:20:27,053 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-59) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:20:27,973 ERROR [org.ovirt.engine.core.bll.SearchQuery] (ajp--127.0.0.1-8702-2) Query SearchQuery failed. Exception message is Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource >2014-01-09 19:20:30,075 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-66) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:20:30,094 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-61) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:20:30,846 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-63) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor77.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:20:30,893 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-62) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:88) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:78) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 19 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 21 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 24 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 27 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 30 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 39 more > >2014-01-09 19:20:33,020 ERROR [org.ovirt.engine.core.bll.SearchQuery] (ajp--127.0.0.1-8702-7) Query SearchQuery failed. Exception message is Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource >2014-01-09 19:20:33,113 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-64) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:20:33,145 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-69) Timer update runtimeinfo failed. Exception:: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:36) [dal.jar:] > at org.ovirt.engine.core.dao.VdsDAODbFacadeImpl.get(VdsDAODbFacadeImpl.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.OnTimer(VdsManager.java:219) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source) [:1.7.0_45] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:137) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 20 more >Caused by: javax.resource.ResourceException: IJ000453: Unable to get managed connection for java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:390) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:464) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:129) > ... 22 more >Caused by: javax.resource.ResourceException: IJ000658: Unexpected throwable while trying to create a connection: null > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:371) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) > at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:329) > ... 25 more >Caused by: javax.resource.ResourceException: Could not create connection > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) > at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) > ... 28 more >Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections. > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:138) > at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) > at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) > at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) > at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) > at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:32) > at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24) > at org.postgresql.Driver.makeConnection(Driver.java:393) > at org.postgresql.Driver.connect(Driver.java:267) > at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) > ... 31 more >Caused by: java.net.ConnectException: Connection refused > at java.net.PlainSocketImpl.socketConnect(Native Method) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) [rt.jar:1.7.0_45] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) [rt.jar:1.7.0_45] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:579) [rt.jar:1.7.0_45] > at java.net.Socket.connect(Socket.java:528) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:425) [rt.jar:1.7.0_45] > at java.net.Socket.<init>(Socket.java:208) [rt.jar:1.7.0_45] > at org.postgresql.core.PGStream.<init>(PGStream.java:62) > at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:76) > ... 40 more > >2014-01-09 19:33:48,980 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-64) Start refreshing all users data >2014-01-09 19:33:49,191 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-67) Attempting to update VMs/Templates Ovf. >2014-01-09 19:33:49,197 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-67) Attempting to update VM OVFs in Data Center CSEnt >2014-01-09 19:33:49,305 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UpdateVMVDSCommand] (DefaultQuartzScheduler_Worker-67) START, UpdateVMVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, storageDomainId = 00000000-0000-0000-0000-000000000000, infoDictionary.size = 3), log id: 1668d910 >2014-01-09 19:33:49,418 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UpdateVMVDSCommand] (DefaultQuartzScheduler_Worker-67) FINISH, UpdateVMVDSCommand, log id: 1668d910 >2014-01-09 19:33:49,423 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-67) Successfully updated VM OVFs in Data Center CSEnt >2014-01-09 19:33:49,424 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-67) Attempting to update template OVFs in Data Center CSEnt >2014-01-09 19:33:49,429 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-67) Successfully updated templates OVFs in Data Center CSEnt >2014-01-09 19:33:49,430 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-67) Attempting to remove unneeded template/vm OVFs in Data Center CSEnt >2014-01-09 19:33:49,435 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.RemoveVMVDSCommand] (DefaultQuartzScheduler_Worker-67) START, RemoveVMVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, storageDomainId = 00000000-0000-0000-0000-000000000000, vmGuid = e48c51ac-dc23-4992-a534-84ed5d1dbca5), log id: 6e93605d >2014-01-09 19:33:49,478 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.RemoveVMVDSCommand] (DefaultQuartzScheduler_Worker-67) FINISH, RemoveVMVDSCommand, log id: 6e93605d >2014-01-09 19:33:49,483 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-67) Successfully removed unneeded template/vm OVFs in Data Center CSEnt >2014-01-09 19:33:51,987 ERROR [org.ovirt.engine.core.bll.adbroker.GetRootDSE] (DefaultQuartzScheduler_Worker-64) Failed to query rootDSE for LDAP server ldap://cytechservices.local:389 due to cytechservices.local:389 >2014-01-09 19:33:51,989 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-64) Failed ldap search server ldap://cytechservices.local:389 using user ryan-a@CYTECHSERVICES.LOCAL due to javax.naming.CommunicationException: cytechservices.local:389 [Root exception is java.net.NoRouteToHostException: No route to host]. We should try the next server >2014-01-09 19:38:48,299 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetIsoListVDSCommand] (pool-6-thread-49) START, GetIsoListVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false), log id: 1e053565 >2014-01-09 19:38:48,302 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HsmGetIsoListVDSCommand] (pool-6-thread-49) START, HsmGetIsoListVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 67610b49 >2014-01-09 19:38:48,490 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HsmGetIsoListVDSCommand] (pool-6-thread-49) FINISH, HsmGetIsoListVDSCommand, return: [virtio-win-0.1-52.iso, centos64x64netinst.iso, virtio-win-0.1-65.iso, winserve2012.iso, kali-linux-1.0.5-amd641.iso, Server2008R2SP1-en-US-July2013.iso, win7.iso, virt-p2v-0.9.1-2.20130730.1.el6.centos.iso, zentyal-3.2-amd64.iso, Elastix-2.4.0-Stable-x86_64-bin-04feb2013.iso, EXCH201064.ISO], log id: 67610b49 >2014-01-09 19:38:48,496 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetIsoListVDSCommand] (pool-6-thread-49) FINISH, GetIsoListVDSCommand, return: [virtio-win-0.1-52.iso, centos64x64netinst.iso, virtio-win-0.1-65.iso, winserve2012.iso, kali-linux-1.0.5-amd641.iso, Server2008R2SP1-en-US-July2013.iso, win7.iso, virt-p2v-0.9.1-2.20130730.1.el6.centos.iso, zentyal-3.2-amd64.iso, Elastix-2.4.0-Stable-x86_64-bin-04feb2013.iso, EXCH201064.ISO], log id: 1e053565 >2014-01-09 19:38:48,506 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (pool-6-thread-49) Finished automatic refresh process for ISO file type with success, for storage domain id de2fccd8-63cc-4662-8878-077b4cfb810e. >2014-01-09 19:45:05,415 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (ajp--127.0.0.1-8702-7) [21d8ec7e] Lock Acquired to object EngineLock [exclusiveLocks= key: 565c5082-b8a3-4946-8ae2-6d3330c8ef9b value: VM >, sharedLocks= ] >2014-01-09 19:45:05,427 WARN [org.ovirt.engine.core.bll.RemoveVmCommand] (ajp--127.0.0.1-8702-7) [21d8ec7e] CanDoAction of action RemoveVm failed. Reasons:VAR__ACTION__REMOVE,VAR__TYPE__VM,ACTION_TYPE_FAILED_DISKS_LOCKED,$diskAliases syslog00_Disk1, syslog00_Disk2 >2014-01-09 19:45:05,428 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (ajp--127.0.0.1-8702-7) [21d8ec7e] Lock freed to object EngineLock [exclusiveLocks= key: 565c5082-b8a3-4946-8ae2-6d3330c8ef9b value: VM >, sharedLocks= ] >2014-01-09 19:45:46,219 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-8) [5abf07a4] Lock Acquired to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 19:45:46,231 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-8) [5abf07a4] CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,ACTION_TYPE_FAILED_VM_IS_DURING_SNAPSHOT >2014-01-09 19:45:46,232 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-8) [5abf07a4] Lock freed to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 19:46:16,005 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-2) [6c7cdfcf] Lock Acquired to object EngineLock [exclusiveLocks= key: be5348a1-85c7-48dd-a8ad-8a0464643bd8 value: VM >, sharedLocks= ] >2014-01-09 19:46:16,025 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-2) [6c7cdfcf] START, IsVmDuringInitiatingVDSCommand( vmId = be5348a1-85c7-48dd-a8ad-8a0464643bd8), log id: 5060d82d >2014-01-09 19:46:16,026 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-2) [6c7cdfcf] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 5060d82d >2014-01-09 19:46:16,066 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-6-thread-48) [6c7cdfcf] Running command: RunVmCommand internal: false. Entities affected : ID: be5348a1-85c7-48dd-a8ad-8a0464643bd8 Type: VM >2014-01-09 19:46:16,091 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-6-thread-48) [6c7cdfcf] START, IsoPrefixVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 2fd8e7d1 >2014-01-09 19:46:16,092 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-6-thread-48) [6c7cdfcf] FINISH, IsoPrefixVDSCommand, return: /rhev/data-center/mnt/192.168.101.101:_iso/de2fccd8-63cc-4662-8878-077b4cfb810e/images/11111111-1111-1111-1111-111111111111, log id: 2fd8e7d1 >2014-01-09 19:46:16,096 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-6-thread-48) [6c7cdfcf] START, CreateVmVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, vm=VM [Apache01]), log id: 539ae2cc >2014-01-09 19:46:16,102 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-48) [6c7cdfcf] START, CreateVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, vm=VM [Apache01]), log id: 7df68777 >2014-01-09 19:46:16,200 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-48) [6c7cdfcf] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand spiceSslCipherSuite=DEFAULT,memSize=8192,kvmEnable=true,smp=4,vmType=kvm,emulatedMachine=rhel6.4.0,keyboardLayout=en-us,memGuaranteedSize=2048,nice=0,display=qxl,smartcardEnable=false,smpCoresPerSocket=1,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,timeOffset=0,transparentHugePages=true,vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8,devices=[Ljava.util.HashMap;@29535c27,acpiEnable=true,vmName=Apache01,cpuType=Nehalem,custom={device_aa0bed01-64aa-4931-9acd-c7adfa14c1dadevice_52bcea7b-5d0b-4a8f-8393-39a24bc6df0b=VmDevice {vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, deviceId=52bcea7b-5d0b-4a8f-8393-39a24bc6df0b, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}, device_aa0bed01-64aa-4931-9acd-c7adfa14c1da=VmDevice {vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, deviceId=aa0bed01-64aa-4931-9acd-c7adfa14c1da, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_aa0bed01-64aa-4931-9acd-c7adfa14c1dadevice_52bcea7b-5d0b-4a8f-8393-39a24bc6df0bdevice_59e96ba0-08c7-43e9-a8ab-357f7afd9fa6device_3bc77c0a-b679-4d81-ba31-ffafbf6f32b2device_6f03e64f-3314-4292-84e5-df6533213eb0=VmDevice {vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, deviceId=6f03e64f-3314-4292-84e5-df6533213eb0, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_aa0bed01-64aa-4931-9acd-c7adfa14c1dadevice_52bcea7b-5d0b-4a8f-8393-39a24bc6df0bdevice_59e96ba0-08c7-43e9-a8ab-357f7afd9fa6device_3bc77c0a-b679-4d81-ba31-ffafbf6f32b2=VmDevice {vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, deviceId=3bc77c0a-b679-4d81-ba31-ffafbf6f32b2, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_aa0bed01-64aa-4931-9acd-c7adfa14c1dadevice_52bcea7b-5d0b-4a8f-8393-39a24bc6df0bdevice_59e96ba0-08c7-43e9-a8ab-357f7afd9fa6=VmDevice {vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, deviceId=59e96ba0-08c7-43e9-a8ab-357f7afd9fa6, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}} >2014-01-09 19:46:16,207 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-48) [6c7cdfcf] FINISH, CreateVDSCommand, log id: 7df68777 >2014-01-09 19:46:16,211 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-6-thread-48) [6c7cdfcf] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 539ae2cc >2014-01-09 19:46:16,211 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-6-thread-48) [6c7cdfcf] Lock freed to object EngineLock [exclusiveLocks= key: be5348a1-85c7-48dd-a8ad-8a0464643bd8 value: VM >, sharedLocks= ] >2014-01-09 19:46:16,219 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) [6c7cdfcf] Correlation ID: 6c7cdfcf, Job ID: 9338a58d-8dfc-4411-a01e-fb91c650fe69, Call Stack: null, Custom Event ID: -1, Message: VM Apache01 was started by admin@internal (Host: ovirt02). >2014-01-09 19:46:19,072 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) [4a6e660e] VM Apache01 be5348a1-85c7-48dd-a8ad-8a0464643bd8 moved from WaitForLaunch --> PoweringUp >2014-01-09 19:46:19,073 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-4) [4a6e660e] START, FullListVdsCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02], vmIds=[be5348a1-85c7-48dd-a8ad-8a0464643bd8]), log id: 543f31cd >2014-01-09 19:46:19,112 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-4) [4a6e660e] FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@deb4f1, log id: 543f31cd >2014-01-09 19:46:25,006 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp--127.0.0.1-8702-13) [1789221a] Running command: SetVmTicketCommand internal: false. Entities affected : ID: be5348a1-85c7-48dd-a8ad-8a0464643bd8 Type: VM >2014-01-09 19:46:25,013 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp--127.0.0.1-8702-13) [1789221a] START, SetVmTicketVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, ticket=GoemGYX3oLZ8, validTime=120,m userName=admin@internal, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 4d815760 >2014-01-09 19:46:25,087 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp--127.0.0.1-8702-13) [1789221a] FINISH, SetVmTicketVDSCommand, log id: 4d815760 >2014-01-09 19:46:25,094 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-13) [1789221a] Correlation ID: 1789221a, Call Stack: null, Custom Event ID: -1, Message: user admin@internal initiated console session for VM Apache01 >2014-01-09 19:47:17,818 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-92) VM Apache01 be5348a1-85c7-48dd-a8ad-8a0464643bd8 moved from PoweringUp --> Up >2014-01-09 19:47:17,836 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-92) Correlation ID: 6c7cdfcf, Job ID: 9338a58d-8dfc-4411-a01e-fb91c650fe69, Call Stack: null, Custom Event ID: -1, Message: VM Apache01 started on Host ovirt02 >2014-01-09 19:47:54,799 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp--127.0.0.1-8702-13) [4f042ab1] Running command: SetVmTicketCommand internal: false. Entities affected : ID: be5348a1-85c7-48dd-a8ad-8a0464643bd8 Type: VM >2014-01-09 19:47:54,803 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp--127.0.0.1-8702-13) [4f042ab1] START, SetVmTicketVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=be5348a1-85c7-48dd-a8ad-8a0464643bd8, ticket=LAd4vixI39j6, validTime=120,m userName=admin@internal, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 590093a2 >2014-01-09 19:47:54,886 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp--127.0.0.1-8702-13) [4f042ab1] FINISH, SetVmTicketVDSCommand, log id: 590093a2 >2014-01-09 19:47:54,892 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-13) [4f042ab1] Correlation ID: 4f042ab1, Call Stack: null, Custom Event ID: -1, Message: user admin@internal initiated console session for VM Apache01 >2014-01-09 19:50:05,958 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-26) VM Apache01 be5348a1-85c7-48dd-a8ad-8a0464643bd8 moved from Up --> RebootInProgress >2014-01-09 19:50:05,966 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-26) START, FullListVdsCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02], vmIds=[be5348a1-85c7-48dd-a8ad-8a0464643bd8]), log id: 529d217e >2014-01-09 19:50:06,001 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-26) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@4a020191, log id: 529d217e >2014-01-09 19:50:57,740 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-2) [4ac56dc6] Lock Acquired to object EngineLock [exclusiveLocks= key: fc077d22-f9a1-45e4-839d-840c128a10c6 value: VM >, sharedLocks= ] >2014-01-09 19:50:57,752 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-2) [4ac56dc6] START, IsVmDuringInitiatingVDSCommand( vmId = fc077d22-f9a1-45e4-839d-840c128a10c6), log id: 7ea6389c >2014-01-09 19:50:57,753 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-2) [4ac56dc6] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 7ea6389c >2014-01-09 19:50:57,775 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-6-thread-49) [4ac56dc6] Running command: RunVmCommand internal: false. Entities affected : ID: fc077d22-f9a1-45e4-839d-840c128a10c6 Type: VM >2014-01-09 19:50:57,808 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-6-thread-49) [4ac56dc6] START, IsoPrefixVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 543fb3c4 >2014-01-09 19:50:57,809 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-6-thread-49) [4ac56dc6] FINISH, IsoPrefixVDSCommand, return: /rhev/data-center/mnt/192.168.101.101:_iso/de2fccd8-63cc-4662-8878-077b4cfb810e/images/11111111-1111-1111-1111-111111111111, log id: 543fb3c4 >2014-01-09 19:50:57,813 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-6-thread-49) [4ac56dc6] START, CreateVmVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=fc077d22-f9a1-45e4-839d-840c128a10c6, vm=VM [Finance]), log id: 49b01a8c >2014-01-09 19:50:57,816 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-49) [4ac56dc6] START, CreateVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=fc077d22-f9a1-45e4-839d-840c128a10c6, vm=VM [Finance]), log id: 44f250e6 >2014-01-09 19:50:57,927 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-49) [4ac56dc6] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand spiceSslCipherSuite=DEFAULT,memSize=2048,kvmEnable=true,smp=2,vmType=kvm,emulatedMachine=rhel6.4.0,keyboardLayout=en-us,memGuaranteedSize=1024,nice=0,display=qxl,smartcardEnable=false,smpCoresPerSocket=1,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,timeOffset=-18004,transparentHugePages=true,vmId=fc077d22-f9a1-45e4-839d-840c128a10c6,devices=[Ljava.util.HashMap;@bb480cd,acpiEnable=true,vmName=Finance,cpuType=Nehalem,custom={device_1954edad-f6c4-4c29-a67d-257a4461d284device_8eb71d17-4848-4e7c-af61-db52bd639e7ddevice_c2086b11-3c93-44f3-aff6-a6f1507c46acdevice_d1fffcc6-4652-461b-80c7-27e672850749device_067278b6-03d4-440e-83fb-8959599f6cdc=VmDevice {vmId=fc077d22-f9a1-45e4-839d-840c128a10c6, deviceId=067278b6-03d4-440e-83fb-8959599f6cdc, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x04, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}, device_1954edad-f6c4-4c29-a67d-257a4461d284device_8eb71d17-4848-4e7c-af61-db52bd639e7d=VmDevice {vmId=fc077d22-f9a1-45e4-839d-840c128a10c6, deviceId=8eb71d17-4848-4e7c-af61-db52bd639e7d, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_1954edad-f6c4-4c29-a67d-257a4461d284device_8eb71d17-4848-4e7c-af61-db52bd639e7ddevice_c2086b11-3c93-44f3-aff6-a6f1507c46acdevice_d1fffcc6-4652-461b-80c7-27e672850749=VmDevice {vmId=fc077d22-f9a1-45e4-839d-840c128a10c6, deviceId=d1fffcc6-4652-461b-80c7-27e672850749, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_1954edad-f6c4-4c29-a67d-257a4461d284device_8eb71d17-4848-4e7c-af61-db52bd639e7ddevice_c2086b11-3c93-44f3-aff6-a6f1507c46ac=VmDevice {vmId=fc077d22-f9a1-45e4-839d-840c128a10c6, deviceId=c2086b11-3c93-44f3-aff6-a6f1507c46ac, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_1954edad-f6c4-4c29-a67d-257a4461d284=VmDevice {vmId=fc077d22-f9a1-45e4-839d-840c128a10c6, deviceId=1954edad-f6c4-4c29-a67d-257a4461d284, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}} >2014-01-09 19:50:57,936 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-49) [4ac56dc6] FINISH, CreateVDSCommand, log id: 44f250e6 >2014-01-09 19:50:57,939 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-6-thread-49) [4ac56dc6] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 49b01a8c >2014-01-09 19:50:57,940 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-6-thread-49) [4ac56dc6] Lock freed to object EngineLock [exclusiveLocks= key: fc077d22-f9a1-45e4-839d-840c128a10c6 value: VM >, sharedLocks= ] >2014-01-09 19:50:57,944 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-49) [4ac56dc6] Correlation ID: 4ac56dc6, Job ID: fae53357-c626-4d32-8763-be12542677b0, Call Stack: null, Custom Event ID: -1, Message: VM Finance was started by admin@internal (Host: ovirt02). >2014-01-09 19:50:58,473 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) [60cb7ef6] VM Apache01 be5348a1-85c7-48dd-a8ad-8a0464643bd8 moved from RebootInProgress --> Up >2014-01-09 19:51:01,548 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-98) VM Finance fc077d22-f9a1-45e4-839d-840c128a10c6 moved from WaitForLaunch --> PoweringUp >2014-01-09 19:51:01,549 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-98) START, FullListVdsCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02], vmIds=[fc077d22-f9a1-45e4-839d-840c128a10c6]), log id: 2beec36c >2014-01-09 19:51:01,585 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-98) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@7c1653cb, log id: 2beec36c >2014-01-09 19:51:01,588 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-98) Received a memballoon Device without an address when processing VM fc077d22-f9a1-45e4-839d-840c128a10c6 devices, skipping device: {alias=balloon0, specParams={model=none}, device=memballoon, type=balloon} >2014-01-09 19:51:58,185 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) VM Finance fc077d22-f9a1-45e4-839d-840c128a10c6 moved from PoweringUp --> Up >2014-01-09 19:51:58,189 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-75) Correlation ID: 4ac56dc6, Job ID: fae53357-c626-4d32-8763-be12542677b0, Call Stack: null, Custom Event ID: -1, Message: VM Finance started on Host ovirt02 >2014-01-09 20:03:49,024 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp--127.0.0.1-8702-12) [5f220027] Running command: SetVmTicketCommand internal: false. Entities affected : ID: fc077d22-f9a1-45e4-839d-840c128a10c6 Type: VM >2014-01-09 20:03:49,030 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp--127.0.0.1-8702-12) [5f220027] START, SetVmTicketVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=fc077d22-f9a1-45e4-839d-840c128a10c6, ticket=e19s8EY4AK1w, validTime=120,m userName=admin@internal, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 5f0c7d4c >2014-01-09 20:03:49,107 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp--127.0.0.1-8702-12) [5f220027] FINISH, SetVmTicketVDSCommand, log id: 5f0c7d4c >2014-01-09 20:03:49,114 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-12) [5f220027] Correlation ID: 5f220027, Call Stack: null, Custom Event ID: -1, Message: user admin@internal initiated console session for VM Finance >2014-01-09 20:17:07,841 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (ajp--127.0.0.1-8702-4) [b907668] Lock Acquired to object EngineLock [exclusiveLocks= key: 565c5082-b8a3-4946-8ae2-6d3330c8ef9b value: VM >, sharedLocks= ] >2014-01-09 20:17:07,882 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (pool-6-thread-48) [b907668] Running command: RemoveVmCommand internal: false. Entities affected : ID: 565c5082-b8a3-4946-8ae2-6d3330c8ef9b Type: VM >2014-01-09 20:17:07,898 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-48) [b907668] START, SetVmStatusVDSCommand( vmId = 565c5082-b8a3-4946-8ae2-6d3330c8ef9b, status = ImageLocked), log id: 3325bdb >2014-01-09 20:17:07,911 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-6-thread-48) [b907668] FINISH, SetVmStatusVDSCommand, log id: 3325bdb >2014-01-09 20:17:07,929 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (pool-6-thread-48) [b907668] Lock freed to object EngineLock [exclusiveLocks= key: 565c5082-b8a3-4946-8ae2-6d3330c8ef9b value: VM >, sharedLocks= ] >2014-01-09 20:17:07,968 INFO [org.ovirt.engine.core.bll.RemoveAllVmImagesCommand] (pool-6-thread-48) [25295d0f] Running command: RemoveAllVmImagesCommand internal: true. Entities affected : ID: 565c5082-b8a3-4946-8ae2-6d3330c8ef9b Type: VM >2014-01-09 20:17:07,994 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (pool-6-thread-48) [257aafc4] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-01-09 20:17:08,032 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (pool-6-thread-48) [257aafc4] START, DeleteImageGroupVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, storageDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, imageGroupId = 3736f6c1-f225-4598-ba79-66dc9236b101, postZeros = false, forceDelete = false), log id: bb51f51 >2014-01-09 20:17:08,688 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (pool-6-thread-48) [257aafc4] Command DeleteImageGroupVDS execution failed. Exception: IrsOperationFailedNoFailoverException: IRSGenericException: IRSErrorException: Image does not exist in domain: 'image=3736f6c1-f225-4598-ba79-66dc9236b101, domain=cbdefcb2-08ca-400f-84e0-1a6c7d28d763' >2014-01-09 20:17:08,690 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (pool-6-thread-48) [257aafc4] FINISH, DeleteImageGroupVDSCommand, log id: bb51f51 >2014-01-09 20:17:08,691 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (pool-6-thread-48) [257aafc4] Disk 3736f6c1-f225-4598-ba79-66dc9236b101 doesnt exist on storage domain cbdefcb2-08ca-400f-84e0-1a6c7d28d763, rolling forward >2014-01-09 20:17:08,762 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (pool-6-thread-48) [1da08068] Running command: RemoveImageCommand internal: true. Entities affected : ID: 00000000-0000-0000-0000-000000000000 Type: Storage >2014-01-09 20:17:08,767 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (pool-6-thread-48) [1da08068] START, DeleteImageGroupVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, storageDomainId = cbdefcb2-08ca-400f-84e0-1a6c7d28d763, imageGroupId = 95b6fcb0-cde9-40e6-a217-6dfbe9693d5f, postZeros = false, forceDelete = false), log id: 49789305 >2014-01-09 20:17:09,371 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (pool-6-thread-48) [1da08068] Command DeleteImageGroupVDS execution failed. Exception: IrsOperationFailedNoFailoverException: IRSGenericException: IRSErrorException: Image does not exist in domain: 'image=95b6fcb0-cde9-40e6-a217-6dfbe9693d5f, domain=cbdefcb2-08ca-400f-84e0-1a6c7d28d763' >2014-01-09 20:17:09,373 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (pool-6-thread-48) [1da08068] FINISH, DeleteImageGroupVDSCommand, log id: 49789305 >2014-01-09 20:17:09,374 INFO [org.ovirt.engine.core.bll.RemoveImageCommand] (pool-6-thread-48) [1da08068] Disk 95b6fcb0-cde9-40e6-a217-6dfbe9693d5f doesnt exist on storage domain cbdefcb2-08ca-400f-84e0-1a6c7d28d763, rolling forward >2014-01-09 20:17:09,396 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-48) [1da08068] Correlation ID: b907668, Job ID: 494b5010-51cb-4067-8e56-4ee01b671f31, Call Stack: null, Custom Event ID: -1, Message: VM syslog00-1 was successfully removed. >2014-01-09 20:17:11,259 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-9) [100212f2] Lock Acquired to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 20:17:11,269 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-9) [100212f2] CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,ACTION_TYPE_FAILED_VM_IS_DURING_SNAPSHOT >2014-01-09 20:17:11,270 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-9) [100212f2] Lock freed to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 20:18:30,296 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-11) [5079b782] Lock Acquired to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 20:18:30,303 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-11) [5079b782] CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,ACTION_TYPE_FAILED_VM_IS_DURING_SNAPSHOT >2014-01-09 20:18:30,305 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-11) [5079b782] Lock freed to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 20:19:16,743 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-3) [1b14411b] Lock Acquired to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 20:19:16,754 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-3) [1b14411b] START, IsVmDuringInitiatingVDSCommand( vmId = 14f2245f-b5a9-48b0-aa78-c8cab04d9be7), log id: 73b82b9b >2014-01-09 20:19:16,756 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-3) [1b14411b] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 73b82b9b >2014-01-09 20:19:16,777 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-6-thread-49) [1b14411b] Running command: RunVmCommand internal: false. Entities affected : ID: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 Type: VM >2014-01-09 20:19:16,806 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-6-thread-49) [1b14411b] START, IsoPrefixVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 4f2b0599 >2014-01-09 20:19:16,808 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.IsoPrefixVDSCommand] (pool-6-thread-49) [1b14411b] FINISH, IsoPrefixVDSCommand, return: /rhev/data-center/mnt/192.168.101.101:_iso/de2fccd8-63cc-4662-8878-077b4cfb810e/images/11111111-1111-1111-1111-111111111111, log id: 4f2b0599 >2014-01-09 20:19:16,814 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-6-thread-49) [1b14411b] START, CreateVmVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=14f2245f-b5a9-48b0-aa78-c8cab04d9be7, vm=VM [syslog00]), log id: 4972200f >2014-01-09 20:19:16,818 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-49) [1b14411b] START, CreateVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=14f2245f-b5a9-48b0-aa78-c8cab04d9be7, vm=VM [syslog00]), log id: 1b40985a >2014-01-09 20:19:16,928 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-49) [1b14411b] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand spiceSslCipherSuite=DEFAULT,memSize=3072,kvmEnable=true,smp=2,vmType=kvm,emulatedMachine=rhel6.4.0,keyboardLayout=en-us,memGuaranteedSize=3072,pitReinjection=false,nice=0,display=qxl,smartcardEnable=false,smpCoresPerSocket=1,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,timeOffset=0,transparentHugePages=true,vmId=14f2245f-b5a9-48b0-aa78-c8cab04d9be7,devices=[Ljava.util.HashMap;@34fadcbe,acpiEnable=true,vmName=syslog00,cpuType=Nehalem,custom={device_7da01459-8cb3-40f2-a369-0fddf723eee7device_54b7688c-fabf-43d9-b65c-e86eef5b913ddevice_b517068f-f4c4-4a8c-bbca-4794fec9bf51device_391ac2eb-da1b-4d03-bc8f-f04f2e4bb835device_dfb5eaab-6f1e-4446-8ce8-4def043c3bab=VmDevice {vmId=14f2245f-b5a9-48b0-aa78-c8cab04d9be7, deviceId=dfb5eaab-6f1e-4446-8ce8-4def043c3bab, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x04, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}, device_7da01459-8cb3-40f2-a369-0fddf723eee7device_54b7688c-fabf-43d9-b65c-e86eef5b913d=VmDevice {vmId=14f2245f-b5a9-48b0-aa78-c8cab04d9be7, deviceId=54b7688c-fabf-43d9-b65c-e86eef5b913d, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_7da01459-8cb3-40f2-a369-0fddf723eee7=VmDevice {vmId=14f2245f-b5a9-48b0-aa78-c8cab04d9be7, deviceId=7da01459-8cb3-40f2-a369-0fddf723eee7, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_7da01459-8cb3-40f2-a369-0fddf723eee7device_54b7688c-fabf-43d9-b65c-e86eef5b913ddevice_b517068f-f4c4-4a8c-bbca-4794fec9bf51device_391ac2eb-da1b-4d03-bc8f-f04f2e4bb835=VmDevice {vmId=14f2245f-b5a9-48b0-aa78-c8cab04d9be7, deviceId=391ac2eb-da1b-4d03-bc8f-f04f2e4bb835, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_7da01459-8cb3-40f2-a369-0fddf723eee7device_54b7688c-fabf-43d9-b65c-e86eef5b913ddevice_b517068f-f4c4-4a8c-bbca-4794fec9bf51=VmDevice {vmId=14f2245f-b5a9-48b0-aa78-c8cab04d9be7, deviceId=b517068f-f4c4-4a8c-bbca-4794fec9bf51, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}} >2014-01-09 20:19:16,947 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-6-thread-49) [1b14411b] FINISH, CreateVDSCommand, log id: 1b40985a >2014-01-09 20:19:16,950 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-6-thread-49) [1b14411b] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 4972200f >2014-01-09 20:19:16,951 INFO [org.ovirt.engine.core.bll.RunVmCommand] (pool-6-thread-49) [1b14411b] Lock freed to object EngineLock [exclusiveLocks= key: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 value: VM >, sharedLocks= ] >2014-01-09 20:19:16,954 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-49) [1b14411b] Correlation ID: 1b14411b, Job ID: 679e8d15-1440-4043-a8e2-c955b48d247b, Call Stack: null, Custom Event ID: -1, Message: VM syslog00 was started by admin@internal (Host: ovirt02). >2014-01-09 20:19:21,563 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) VM syslog00 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 moved from WaitForLaunch --> PoweringUp >2014-01-09 20:19:21,564 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-50) START, FullListVdsCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vds=Host[ovirt02], vmIds=[14f2245f-b5a9-48b0-aa78-c8cab04d9be7]), log id: 42618a6c >2014-01-09 20:19:21,602 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-50) FINISH, FullListVdsCommand, return: [Ljava.util.HashMap;@11c6da80, log id: 42618a6c >2014-01-09 20:20:17,476 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-31) VM syslog00 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 moved from PoweringUp --> Up >2014-01-09 20:20:17,482 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-31) Correlation ID: 1b14411b, Job ID: 679e8d15-1440-4043-a8e2-c955b48d247b, Call Stack: null, Custom Event ID: -1, Message: VM syslog00 started on Host ovirt02 >2014-01-09 20:26:52,626 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) VM Apache01 be5348a1-85c7-48dd-a8ad-8a0464643bd8 moved from Up --> RebootInProgress >2014-01-09 20:26:59,711 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp--127.0.0.1-8702-1) [5a8de62f] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 Type: VM >2014-01-09 20:26:59,716 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp--127.0.0.1-8702-1) [5a8de62f] START, SetVmTicketVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=14f2245f-b5a9-48b0-aa78-c8cab04d9be7, ticket=4qL9TfMefxNF, validTime=120,m userName=admin@internal, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 21d4ad84 >2014-01-09 20:26:59,799 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp--127.0.0.1-8702-1) [5a8de62f] FINISH, SetVmTicketVDSCommand, log id: 21d4ad84 >2014-01-09 20:26:59,813 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-1) [5a8de62f] Correlation ID: 5a8de62f, Call Stack: null, Custom Event ID: -1, Message: user admin@internal initiated console session for VM syslog00 >2014-01-09 20:27:05,148 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp--127.0.0.1-8702-11) [397c82ba] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 Type: VM >2014-01-09 20:27:05,153 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp--127.0.0.1-8702-11) [397c82ba] START, SetVmTicketVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=14f2245f-b5a9-48b0-aa78-c8cab04d9be7, ticket=ibLKU4qYz9dp, validTime=120,m userName=admin@internal, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 1dff32b4 >2014-01-09 20:27:05,273 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp--127.0.0.1-8702-11) [397c82ba] FINISH, SetVmTicketVDSCommand, log id: 1dff32b4 >2014-01-09 20:27:05,279 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-11) [397c82ba] Correlation ID: 397c82ba, Call Stack: null, Custom Event ID: -1, Message: user admin@internal initiated console session for VM syslog00 >2014-01-09 20:27:54,522 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-59) VM Apache01 be5348a1-85c7-48dd-a8ad-8a0464643bd8 moved from RebootInProgress --> Up >2014-01-09 20:33:49,483 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-54) Attempting to update VMs/Templates Ovf. >2014-01-09 20:33:49,485 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-54) Attempting to update VM OVFs in Data Center CSEnt >2014-01-09 20:33:49,493 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-54) Successfully updated VM OVFs in Data Center CSEnt >2014-01-09 20:33:49,495 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-54) Attempting to update template OVFs in Data Center CSEnt >2014-01-09 20:33:49,497 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-54) Successfully updated templates OVFs in Data Center CSEnt >2014-01-09 20:33:49,498 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-54) Attempting to remove unneeded template/vm OVFs in Data Center CSEnt >2014-01-09 20:33:49,500 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.RemoveVMVDSCommand] (DefaultQuartzScheduler_Worker-54) START, RemoveVMVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false, storageDomainId = 00000000-0000-0000-0000-000000000000, vmGuid = 565c5082-b8a3-4946-8ae2-6d3330c8ef9b), log id: 3db5c3f7 >2014-01-09 20:33:49,561 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.RemoveVMVDSCommand] (DefaultQuartzScheduler_Worker-54) FINISH, RemoveVMVDSCommand, log id: 3db5c3f7 >2014-01-09 20:33:49,564 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-54) Successfully removed unneeded template/vm OVFs in Data Center CSEnt >2014-01-09 20:33:51,991 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-55) Start refreshing all users data >2014-01-09 20:33:52,166 ERROR [org.ovirt.engine.core.bll.adbroker.LdapSearchExceptionHandler] (DefaultQuartzScheduler_Worker-55) Ldap authentication failed. Please check that the login name , password and path are correct. >2014-01-09 20:33:52,168 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-55) Failed ldap search server ldap://cytechservices.local:389 using user ryan-a@CYTECHSERVICES.LOCAL due to Kerberos error. Please check log for further details.. We should not try the next server >2014-01-09 20:37:59,720 ERROR [org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy] (ajp--127.0.0.1-8702-1) Kerberos error: cydc-01.cytechservices.local. >2014-01-09 20:37:59,722 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (ajp--127.0.0.1-8702-1) Failed ldap search server ldap://cytechservices.local:389 using user patrick.denton@CYTECHSERVICES.LOCAL due to Kerberos error. Please check log for further details.. We should not try the next server >2014-01-09 20:37:59,724 ERROR [org.ovirt.engine.core.bll.adbroker.LdapAuthenticateUserCommand] (ajp--127.0.0.1-8702-1) Failed authenticating user: patrick.denton to domain cytechservices.local. Ldap Query Type is getUserByName >2014-01-09 20:37:59,726 ERROR [org.ovirt.engine.core.bll.adbroker.LdapAuthenticateUserCommand] (ajp--127.0.0.1-8702-1) Kerberos error. Please check log for further details. >2014-01-09 20:37:59,727 ERROR [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-1) USER_FAILED_TO_AUTHENTICATE : patrick.denton >2014-01-09 20:37:59,728 WARN [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-1) CanDoAction of action LoginAdminUser failed. Reasons:USER_FAILED_TO_AUTHENTICATE >2014-01-09 20:38:45,825 ERROR [org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy] (ajp--127.0.0.1-8702-2) Kerberos error: cydc-02.cytechservices.local. >2014-01-09 20:38:45,826 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (ajp--127.0.0.1-8702-2) Failed ldap search server ldap://cytechservices.local:389 using user admin@CYTECHSERVICES.LOCAL due to Kerberos error. Please check log for further details.. We should not try the next server >2014-01-09 20:38:45,828 ERROR [org.ovirt.engine.core.bll.adbroker.LdapAuthenticateUserCommand] (ajp--127.0.0.1-8702-2) Failed authenticating user: admin to domain cytechservices.local. Ldap Query Type is getUserByName >2014-01-09 20:38:45,829 ERROR [org.ovirt.engine.core.bll.adbroker.LdapAuthenticateUserCommand] (ajp--127.0.0.1-8702-2) Kerberos error. Please check log for further details. >2014-01-09 20:38:45,830 ERROR [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-2) USER_FAILED_TO_AUTHENTICATE : admin >2014-01-09 20:38:45,831 WARN [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-2) CanDoAction of action LoginAdminUser failed. Reasons:USER_FAILED_TO_AUTHENTICATE >2014-01-09 20:38:47,288 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp--127.0.0.1-8702-10) [4e14945a] Running command: LogoutUserCommand internal: false. >2014-01-09 20:38:47,292 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-10) [4e14945a] Correlation ID: 4e14945a, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2014-01-09 20:38:48,515 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetIsoListVDSCommand] (pool-6-thread-48) START, GetIsoListVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false), log id: 98f7cfc >2014-01-09 20:38:48,517 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HsmGetIsoListVDSCommand] (pool-6-thread-48) START, HsmGetIsoListVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 2b56894b >2014-01-09 20:38:48,582 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HsmGetIsoListVDSCommand] (pool-6-thread-48) FINISH, HsmGetIsoListVDSCommand, return: [virtio-win-0.1-52.iso, centos64x64netinst.iso, virtio-win-0.1-65.iso, winserve2012.iso, kali-linux-1.0.5-amd641.iso, Server2008R2SP1-en-US-July2013.iso, win7.iso, virt-p2v-0.9.1-2.20130730.1.el6.centos.iso, zentyal-3.2-amd64.iso, Elastix-2.4.0-Stable-x86_64-bin-04feb2013.iso, EXCH201064.ISO], log id: 2b56894b >2014-01-09 20:38:48,585 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetIsoListVDSCommand] (pool-6-thread-48) FINISH, GetIsoListVDSCommand, return: [virtio-win-0.1-52.iso, centos64x64netinst.iso, virtio-win-0.1-65.iso, winserve2012.iso, kali-linux-1.0.5-amd641.iso, Server2008R2SP1-en-US-July2013.iso, win7.iso, virt-p2v-0.9.1-2.20130730.1.el6.centos.iso, zentyal-3.2-amd64.iso, Elastix-2.4.0-Stable-x86_64-bin-04feb2013.iso, EXCH201064.ISO], log id: 98f7cfc >2014-01-09 20:38:48,593 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (pool-6-thread-48) Finished automatic refresh process for ISO file type with success, for storage domain id de2fccd8-63cc-4662-8878-077b4cfb810e. >2014-01-09 20:38:55,707 INFO [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-7) Running command: LoginAdminUserCommand internal: false. >2014-01-09 20:38:55,712 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2014-01-09 20:38:55,932 INFO [org.ovirt.engine.core.bll.LoginUserCommand] (ajp--127.0.0.1-8702-4) Running command: LoginUserCommand internal: false. >2014-01-09 20:38:55,936 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in. >2014-01-09 20:41:29,263 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp--127.0.0.1-8702-8) [f76fbf6] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 14f2245f-b5a9-48b0-aa78-c8cab04d9be7 Type: VM >2014-01-09 20:41:29,268 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp--127.0.0.1-8702-8) [f76fbf6] START, SetVmTicketVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, vmId=14f2245f-b5a9-48b0-aa78-c8cab04d9be7, ticket=/H8MdvgkM1b9, validTime=120,m userName=admin@internal, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 74607352 >2014-01-09 20:41:29,342 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp--127.0.0.1-8702-8) [f76fbf6] FINISH, SetVmTicketVDSCommand, log id: 74607352 >2014-01-09 20:41:29,349 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-8) [f76fbf6] Correlation ID: f76fbf6, Call Stack: null, Custom Event ID: -1, Message: user admin@internal initiated console session for VM syslog00 >2014-01-09 20:58:45,471 ERROR [org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy] (ajp--127.0.0.1-8702-3) Kerberos error: cydc-01.cytechservices.local. >2014-01-09 20:58:45,473 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (ajp--127.0.0.1-8702-3) Failed ldap search server ldap://cytechservices.local:389 using user ryan-a@CYTECHSERVICES.LOCAL due to Kerberos error. Please check log for further details.. We should not try the next server >2014-01-09 20:58:45,475 ERROR [org.ovirt.engine.core.bll.adbroker.LdapAuthenticateUserCommand] (ajp--127.0.0.1-8702-3) Failed authenticating user: ryan-a to domain cytechservices.local. Ldap Query Type is getUserByName >2014-01-09 20:58:45,477 ERROR [org.ovirt.engine.core.bll.adbroker.LdapAuthenticateUserCommand] (ajp--127.0.0.1-8702-3) Kerberos error. Please check log for further details. >2014-01-09 20:58:45,478 ERROR [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-3) USER_FAILED_TO_AUTHENTICATE : ryan-a >2014-01-09 20:58:45,479 WARN [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-3) CanDoAction of action LoginAdminUser failed. Reasons:USER_FAILED_TO_AUTHENTICATE >2014-01-09 21:04:17,948 ERROR [org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy] (ajp--127.0.0.1-8702-12) Kerberos error: cydc-01.cytechservices.local. >2014-01-09 21:04:17,950 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (ajp--127.0.0.1-8702-12) Failed ldap search server ldap://cytechservices.local:389 using user ryan-a@CYTECHSERVICES.LOCAL due to Kerberos error. Please check log for further details.. We should not try the next server >2014-01-09 21:04:17,952 ERROR [org.ovirt.engine.core.bll.adbroker.LdapAuthenticateUserCommand] (ajp--127.0.0.1-8702-12) Failed authenticating user: ryan-a to domain cytechservices.local. Ldap Query Type is getUserByName >2014-01-09 21:04:17,953 ERROR [org.ovirt.engine.core.bll.adbroker.LdapAuthenticateUserCommand] (ajp--127.0.0.1-8702-12) Kerberos error. Please check log for further details. >2014-01-09 21:04:17,954 ERROR [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-12) USER_FAILED_TO_AUTHENTICATE : ryan-a >2014-01-09 21:04:17,955 WARN [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-12) CanDoAction of action LoginAdminUser failed. Reasons:USER_FAILED_TO_AUTHENTICATE >2014-01-09 21:04:54,006 ERROR [org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy] (ajp--127.0.0.1-8702-4) Kerberos error: cydc-02.cytechservices.local. >2014-01-09 21:04:54,007 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (ajp--127.0.0.1-8702-4) Failed ldap search server ldap://cytechservices.local:389 using user ryan-a@CYTECHSERVICES.LOCAL due to Kerberos error. Please check log for further details.. We should not try the next server >2014-01-09 21:04:54,009 ERROR [org.ovirt.engine.core.bll.adbroker.LdapAuthenticateUserCommand] (ajp--127.0.0.1-8702-4) Failed authenticating user: ryan-a to domain cytechservices.local. Ldap Query Type is getUserByName >2014-01-09 21:04:54,010 ERROR [org.ovirt.engine.core.bll.adbroker.LdapAuthenticateUserCommand] (ajp--127.0.0.1-8702-4) Kerberos error. Please check log for further details. >2014-01-09 21:04:54,011 ERROR [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-4) USER_FAILED_TO_AUTHENTICATE : ryan-a >2014-01-09 21:04:54,012 WARN [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-4) CanDoAction of action LoginAdminUser failed. Reasons:USER_FAILED_TO_AUTHENTICATE >2014-01-09 21:33:49,565 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf. >2014-01-09 21:33:49,567 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VM OVFs in Data Center CSEnt >2014-01-09 21:33:49,576 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Successfully updated VM OVFs in Data Center CSEnt >2014-01-09 21:33:49,577 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update template OVFs in Data Center CSEnt >2014-01-09 21:33:49,580 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Successfully updated templates OVFs in Data Center CSEnt >2014-01-09 21:33:49,581 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to remove unneeded template/vm OVFs in Data Center CSEnt >2014-01-09 21:33:49,583 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Successfully removed unneeded template/vm OVFs in Data Center CSEnt >2014-01-09 21:33:52,171 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-27) Start refreshing all users data >2014-01-09 21:33:52,210 ERROR [org.ovirt.engine.core.bll.adbroker.LdapSearchExceptionHandler] (DefaultQuartzScheduler_Worker-27) Ldap authentication failed. Please check that the login name , password and path are correct. >2014-01-09 21:33:52,212 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-27) Failed ldap search server ldap://cytechservices.local:389 using user ryan-a@CYTECHSERVICES.LOCAL due to Kerberos error. Please check log for further details.. We should not try the next server >2014-01-09 21:33:52,874 INFO [org.ovirt.engine.core.bll.LogoutUserCommand] (ajp--127.0.0.1-8702-7) [74ca18c1] Running command: LogoutUserCommand internal: false. >2014-01-09 21:33:52,879 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-7) [74ca18c1] Correlation ID: 74ca18c1, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged out. >2014-01-09 21:38:48,601 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetIsoListVDSCommand] (pool-6-thread-50) START, GetIsoListVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false), log id: 4621ac8b >2014-01-09 21:38:48,604 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HsmGetIsoListVDSCommand] (pool-6-thread-50) START, HsmGetIsoListVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 4cc6fda5 >2014-01-09 21:38:48,673 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HsmGetIsoListVDSCommand] (pool-6-thread-50) FINISH, HsmGetIsoListVDSCommand, return: [virtio-win-0.1-52.iso, centos64x64netinst.iso, virtio-win-0.1-65.iso, winserve2012.iso, kali-linux-1.0.5-amd641.iso, Server2008R2SP1-en-US-July2013.iso, win7.iso, virt-p2v-0.9.1-2.20130730.1.el6.centos.iso, zentyal-3.2-amd64.iso, Elastix-2.4.0-Stable-x86_64-bin-04feb2013.iso, EXCH201064.ISO], log id: 4cc6fda5 >2014-01-09 21:38:48,675 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetIsoListVDSCommand] (pool-6-thread-50) FINISH, GetIsoListVDSCommand, return: [virtio-win-0.1-52.iso, centos64x64netinst.iso, virtio-win-0.1-65.iso, winserve2012.iso, kali-linux-1.0.5-amd641.iso, Server2008R2SP1-en-US-July2013.iso, win7.iso, virt-p2v-0.9.1-2.20130730.1.el6.centos.iso, zentyal-3.2-amd64.iso, Elastix-2.4.0-Stable-x86_64-bin-04feb2013.iso, EXCH201064.ISO], log id: 4621ac8b >2014-01-09 21:38:48,683 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (pool-6-thread-50) Finished automatic refresh process for ISO file type with success, for storage domain id de2fccd8-63cc-4662-8878-077b4cfb810e. >2014-01-09 22:33:49,584 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-14) Attempting to update VMs/Templates Ovf. >2014-01-09 22:33:49,586 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-14) Attempting to update VM OVFs in Data Center CSEnt >2014-01-09 22:33:49,594 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-14) Successfully updated VM OVFs in Data Center CSEnt >2014-01-09 22:33:49,595 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-14) Attempting to update template OVFs in Data Center CSEnt >2014-01-09 22:33:49,598 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-14) Successfully updated templates OVFs in Data Center CSEnt >2014-01-09 22:33:49,599 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-14) Attempting to remove unneeded template/vm OVFs in Data Center CSEnt >2014-01-09 22:33:49,600 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-14) Successfully removed unneeded template/vm OVFs in Data Center CSEnt >2014-01-09 22:33:52,215 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-18) Start refreshing all users data >2014-01-09 22:33:52,222 ERROR [org.ovirt.engine.core.bll.adbroker.LdapSearchExceptionHandler] (DefaultQuartzScheduler_Worker-18) Error in communicating with LDAP server cytechservices.local:389; nested exception is javax.naming.CommunicationException: cytechservices.local:389 [Root exception is java.net.ConnectException: Connection refused] >2014-01-09 22:33:52,227 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-18) Failed ldap search server ldap://cytechservices.local:389 using user ryan-a@CYTECHSERVICES.LOCAL due to cytechservices.local:389; nested exception is javax.naming.CommunicationException: cytechservices.local:389 [Root exception is java.net.ConnectException: Connection refused]. We should try the next server >2014-01-09 22:38:48,688 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetIsoListVDSCommand] (pool-6-thread-48) START, GetIsoListVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false), log id: 5b2e88cb >2014-01-09 22:38:48,690 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HsmGetIsoListVDSCommand] (pool-6-thread-48) START, HsmGetIsoListVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 4c16f631 >2014-01-09 22:38:48,748 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HsmGetIsoListVDSCommand] (pool-6-thread-48) FINISH, HsmGetIsoListVDSCommand, return: [virtio-win-0.1-52.iso, centos64x64netinst.iso, virtio-win-0.1-65.iso, winserve2012.iso, kali-linux-1.0.5-amd641.iso, Server2008R2SP1-en-US-July2013.iso, win7.iso, virt-p2v-0.9.1-2.20130730.1.el6.centos.iso, zentyal-3.2-amd64.iso, Elastix-2.4.0-Stable-x86_64-bin-04feb2013.iso, EXCH201064.ISO], log id: 4c16f631 >2014-01-09 22:38:48,750 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetIsoListVDSCommand] (pool-6-thread-48) FINISH, GetIsoListVDSCommand, return: [virtio-win-0.1-52.iso, centos64x64netinst.iso, virtio-win-0.1-65.iso, winserve2012.iso, kali-linux-1.0.5-amd641.iso, Server2008R2SP1-en-US-July2013.iso, win7.iso, virt-p2v-0.9.1-2.20130730.1.el6.centos.iso, zentyal-3.2-amd64.iso, Elastix-2.4.0-Stable-x86_64-bin-04feb2013.iso, EXCH201064.ISO], log id: 5b2e88cb >2014-01-09 22:38:48,756 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (pool-6-thread-48) Finished automatic refresh process for ISO file type with success, for storage domain id de2fccd8-63cc-4662-8878-077b4cfb810e. >2014-01-09 23:33:49,601 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-96) [60cb7ef6] Attempting to update VMs/Templates Ovf. >2014-01-09 23:33:49,603 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-96) [60cb7ef6] Attempting to update VM OVFs in Data Center CSEnt >2014-01-09 23:33:49,605 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-96) [60cb7ef6] Successfully updated VM OVFs in Data Center CSEnt >2014-01-09 23:33:49,605 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-96) [60cb7ef6] Attempting to update template OVFs in Data Center CSEnt >2014-01-09 23:33:49,606 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-96) [60cb7ef6] Successfully updated templates OVFs in Data Center CSEnt >2014-01-09 23:33:49,607 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-96) [60cb7ef6] Attempting to remove unneeded template/vm OVFs in Data Center CSEnt >2014-01-09 23:33:49,608 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-96) [60cb7ef6] Successfully removed unneeded template/vm OVFs in Data Center CSEnt >2014-01-09 23:33:52,230 INFO [org.ovirt.engine.core.bll.DbUserCacheManager] (DefaultQuartzScheduler_Worker-10) Start refreshing all users data >2014-01-09 23:33:52,260 ERROR [org.ovirt.engine.core.bll.adbroker.LdapSearchExceptionHandler] (DefaultQuartzScheduler_Worker-10) Ldap authentication failed. Please check that the login name , password and path are correct. >2014-01-09 23:33:52,263 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (DefaultQuartzScheduler_Worker-10) Failed ldap search server ldap://cytechservices.local:389 using user ryan-a@CYTECHSERVICES.LOCAL due to Kerberos error. Please check log for further details.. We should not try the next server >2014-01-09 23:38:48,759 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetIsoListVDSCommand] (pool-6-thread-49) START, GetIsoListVDSCommand( storagePoolId = 55d2a7cd-2925-40e7-b6c8-73c0f6d5084d, ignoreFailoverLimit = false), log id: 5b39dd26 >2014-01-09 23:38:48,762 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HsmGetIsoListVDSCommand] (pool-6-thread-49) START, HsmGetIsoListVDSCommand(HostName = ovirt02, HostId = bf9ab67d-b589-4c9f-8ef5-d2fc9f9bd0d5, storagePoolId=55d2a7cd-2925-40e7-b6c8-73c0f6d5084d), log id: 33e61894 >2014-01-09 23:38:48,821 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HsmGetIsoListVDSCommand] (pool-6-thread-49) FINISH, HsmGetIsoListVDSCommand, return: [virtio-win-0.1-52.iso, centos64x64netinst.iso, virtio-win-0.1-65.iso, winserve2012.iso, kali-linux-1.0.5-amd641.iso, Server2008R2SP1-en-US-July2013.iso, win7.iso, virt-p2v-0.9.1-2.20130730.1.el6.centos.iso, zentyal-3.2-amd64.iso, Elastix-2.4.0-Stable-x86_64-bin-04feb2013.iso, EXCH201064.ISO], log id: 33e61894 >2014-01-09 23:38:48,823 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetIsoListVDSCommand] (pool-6-thread-49) FINISH, GetIsoListVDSCommand, return: [virtio-win-0.1-52.iso, centos64x64netinst.iso, virtio-win-0.1-65.iso, winserve2012.iso, kali-linux-1.0.5-amd641.iso, Server2008R2SP1-en-US-July2013.iso, win7.iso, virt-p2v-0.9.1-2.20130730.1.el6.centos.iso, zentyal-3.2-amd64.iso, Elastix-2.4.0-Stable-x86_64-bin-04feb2013.iso, EXCH201064.ISO], log id: 5b39dd26 >2014-01-09 23:38:48,828 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (pool-6-thread-49) Finished automatic refresh process for ISO file type with success, for storage domain id de2fccd8-63cc-4662-8878-077b4cfb810e. >2014-01-10 00:33:49,610 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-91) Attempting to update VMs/Templates Ovf.
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Raw
Actions:
View
Attachments on
bug 1039115
: 851683