Attachment 621747 Details for Bug 863211 – engine-drop

engine-drop

engine-drop.log (text/x-log), 46.62 KB, created by Pavel Stehlik on 2012-10-04 17:15:34 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Pavel Stehlik

Created: 2012-10-04 17:15:34 UTC

Size: 46.62 KB

patch

obsolete

>2012-10-04 10:15:00,001 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-38) Autorecovering hosts is disabled, skipping
>2012-10-04 10:15:00,001 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-38) Autorecovering storage domains is disabled, skipping
>2012-10-04 10:15:18,754 INFO  [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (QuartzScheduler_Worker-24) [384ef08f] Running command: InitVdsOnUpCommand internal: true.
>2012-10-04 10:15:18,875 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (QuartzScheduler_Worker-24) [450a07c9] START, ValidateStorageServerConnectionVDSCommand(HostName = slot-7, HostId = 0b5eb534-08fc-11e2-a34d-001a4a013f65, storagePoolId = 40512d26-08a2-11e2-8a0b-001a4a013f65, storageType = NFS, connectionList = [{ id: 6620568a-08a2-11e2-a7f2-001a4a013f65, connection: ps-ad2-set.rhev.lab.eng.brq.redhat.com:/var/lib/exports/iso3, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: 220cea3f-3920-4bbc-ba84-f36eb99efaca, connection: 10.34.63.202:/mnt/export/nfs/lv1/pstehlik/nfs01, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 1e6a1d02
>2012-10-04 10:15:18,904 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (QuartzScheduler_Worker-24) [450a07c9] FINISH, ValidateStorageServerConnectionVDSCommand, return: {6620568a-08a2-11e2-a7f2-001a4a013f65=0, 220cea3f-3920-4bbc-ba84-f36eb99efaca=0}, log id: 1e6a1d02
>2012-10-04 10:15:18,904 INFO  [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (QuartzScheduler_Worker-24) [450a07c9] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected :  ID: 40512d26-08a2-11e2-8a0b-001a4a013f65 Type: StoragePool
>2012-10-04 10:15:18,923 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (QuartzScheduler_Worker-24) [450a07c9] START, ConnectStorageServerVDSCommand(HostName = slot-7, HostId = 0b5eb534-08fc-11e2-a34d-001a4a013f65, storagePoolId = 40512d26-08a2-11e2-8a0b-001a4a013f65, storageType = NFS, connectionList = [{ id: 6620568a-08a2-11e2-a7f2-001a4a013f65, connection: ps-ad2-set.rhev.lab.eng.brq.redhat.com:/var/lib/exports/iso3, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: 220cea3f-3920-4bbc-ba84-f36eb99efaca, connection: 10.34.63.202:/mnt/export/nfs/lv1/pstehlik/nfs01, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 2417e122
>2012-10-04 10:15:18,962 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (QuartzScheduler_Worker-24) [450a07c9] FINISH, ConnectStorageServerVDSCommand, return: {6620568a-08a2-11e2-a7f2-001a4a013f65=0, 220cea3f-3920-4bbc-ba84-f36eb99efaca=0}, log id: 2417e122
>2012-10-04 10:15:18,962 INFO  [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (QuartzScheduler_Worker-24) [450a07c9] Host slot-7 storage connection was succeeded 
>2012-10-04 10:15:19,004 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-24) [450a07c9] START, ConnectStoragePoolVDSCommand(HostName = slot-7, HostId = 0b5eb534-08fc-11e2-a34d-001a4a013f65, storagePoolId = 40512d26-08a2-11e2-8a0b-001a4a013f65, vds_spm_id = 1, masterDomainId = 97d9bd45-2028-47d5-a1a2-b9c1d3bdb421, masterVersion = 1), log id: 65561c1a
>2012-10-04 10:15:22,330 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-24) [450a07c9] FINISH, ConnectStoragePoolVDSCommand, log id: 65561c1a
>2012-10-04 10:15:22,420 INFO  [org.ovirt.engine.core.bll.FencingExecutor] (QuartzScheduler_Worker-24) [450a07c9] Executing <Status> Power Management command, Proxy Host:srh-211, Agent:ipmilan, Target Host:slot-7, Management IP:dell-pem1000e-01-bl7.mgmt.eng.brq.redhat.com, User:root, Options:
>2012-10-04 10:15:22,466 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (QuartzScheduler_Worker-24) [450a07c9] START, FenceVdsVDSCommand(HostName = srh-211, HostId = 04be734c-0d41-11e2-a7ed-001a4a013f65, targetVdsId = 0b5eb534-08fc-11e2-a34d-001a4a013f65, action = Status, ip = dell-pem1000e-01-bl7.mgmt.eng.brq.redhat.com, port = , type = ipmilan, user = root, password = ******, options = ''), log id: edfff8c
>2012-10-04 10:15:22,613 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (QuartzScheduler_Worker-24) [450a07c9] FINISH, FenceVdsVDSCommand, return: Test Succeeded, Host Status is: on, log id: edfff8c
>2012-10-04 10:15:22,620 INFO  [org.ovirt.engine.core.bll.MultipleActionsRunner] (pool-4-thread-50) [450a07c9] MultipleActionsRunner of type MigrateVmToServer invoked with no actions
>2012-10-04 10:15:22,642 INFO  [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (QuartzScheduler_Worker-24) [3cdab514] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected :  ID: 0b5eb534-08fc-11e2-a34d-001a4a013f65 Type: VDS
>2012-10-04 10:15:22,666 INFO  [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (QuartzScheduler_Worker-24) [3acf8fe2] Running command: HandleVdsVersionCommand internal: true. Entities affected :  ID: 0b5eb534-08fc-11e2-a34d-001a4a013f65 Type: VDS
>2012-10-04 10:18:57,642 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-60) XML RPC error in command GetStatsVDS ( HostName = slot-7 ), the error was: java.util.concurrent.TimeoutException, TimeoutException:  
>2012-10-04 10:18:57,643 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (QuartzScheduler_Worker-60) vds::refreshVdsStats Failed getVdsStats,  vds = 0b5eb534-08fc-11e2-a34d-001a4a013f65 : slot-7, error = VDSNetworkException: VDSNetworkException: 
>2012-10-04 10:18:57,646 WARN  [org.ovirt.engine.core.vdsbroker.VdsManager] (QuartzScheduler_Worker-60) ResourceManager::refreshVdsRunTimeInfo::Failed to refresh VDS , vds = 0b5eb534-08fc-11e2-a34d-001a4a013f65 : slot-7, VDS Network Error, continuing.
>VDSNetworkException: 
>2012-10-04 10:20:00,001 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-47) Autorecovering hosts is disabled, skipping
>2012-10-04 10:20:00,001 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-47) Autorecovering storage domains is disabled, skipping
>2012-10-04 10:21:59,673 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-74) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.TimeoutException, TimeoutException:  
>2012-10-04 10:21:59,674 WARN  [org.ovirt.engine.core.vdsbroker.VdsManager] (QuartzScheduler_Worker-74) ResourceManager::refreshVdsRunTimeInfo::Failed to refresh VDS , vds = 0b5eb534-08fc-11e2-a34d-001a4a013f65 : slot-7, VDS Network Error, continuing.
>VDSNetworkException: 
>2012-10-04 10:25:00,001 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-5) Autorecovering hosts is disabled, skipping
>2012-10-04 10:25:00,001 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-5) Autorecovering storage domains is disabled, skipping
>2012-10-04 10:25:01,699 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-79) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.TimeoutException, TimeoutException:  
>2012-10-04 10:25:01,699 ERROR [org.ovirt.engine.core.vdsbroker.VdsManager] (QuartzScheduler_Worker-79) VDS::handleNetworkException Server failed to respond,  vds_id = 0b5eb534-08fc-11e2-a34d-001a4a013f65, vds_name = slot-7, error = VDSNetworkException: 
>2012-10-04 10:25:01,725 INFO  [org.ovirt.engine.core.bll.VdsEventListener] (pool-4-thread-50) ResourceManager::vdsNotResponding entered for Host 0b5eb534-08fc-11e2-a34d-001a4a013f65, 10.34.63.137
>2012-10-04 10:25:01,800 INFO  [org.ovirt.engine.core.bll.FencingExecutor] (pool-4-thread-50) Executing <Status> Power Management command, Proxy Host:srh-211, Agent:ipmilan, Target Host:slot-7, Management IP:dell-pem1000e-01-bl7.mgmt.eng.brq.redhat.com, User:root, Options:
>2012-10-04 10:25:01,801 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (pool-4-thread-50) START, FenceVdsVDSCommand(HostName = srh-211, HostId = 04be734c-0d41-11e2-a7ed-001a4a013f65, targetVdsId = 0b5eb534-08fc-11e2-a34d-001a4a013f65, action = Status, ip = dell-pem1000e-01-bl7.mgmt.eng.brq.redhat.com, port = , type = ipmilan, user = root, password = ******, options = ''), log id: d93775
>2012-10-04 10:25:02,065 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (pool-4-thread-50) FINISH, FenceVdsVDSCommand, return: Test Succeeded, Host Status is: on, log id: d93775
>2012-10-04 10:25:02,072 INFO  [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (pool-4-thread-50) Running command: VdsNotRespondingTreatmentCommand internal: true. Entities affected :  ID: 0b5eb534-08fc-11e2-a34d-001a4a013f65 Type: VDS
>2012-10-04 10:25:02,125 INFO  [org.ovirt.engine.core.bll.StopVdsCommand] (pool-4-thread-50) [46f55832] Running command: StopVdsCommand internal: true. Entities affected :  ID: 0b5eb534-08fc-11e2-a34d-001a4a013f65 Type: VDS
>2012-10-04 10:25:02,144 INFO  [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-50) [46f55832] START, SetVdsStatusVDSCommand(HostName = slot-7, HostId = 0b5eb534-08fc-11e2-a34d-001a4a013f65, status=Reboot, nonOperationalReason=NONE), log id: 2d8753c4
>2012-10-04 10:25:02,168 INFO  [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-50) [46f55832] FINISH, SetVdsStatusVDSCommand, log id: 2d8753c4
>2012-10-04 10:25:02,262 INFO  [org.ovirt.engine.core.bll.FencingExecutor] (pool-4-thread-50) [46f55832] Executing <Stop> Power Management command, Proxy Host:srh-211, Agent:ipmilan, Target Host:slot-7, Management IP:dell-pem1000e-01-bl7.mgmt.eng.brq.redhat.com, User:root, Options:
>2012-10-04 10:25:02,264 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (pool-4-thread-50) [46f55832] START, FenceVdsVDSCommand(HostName = srh-211, HostId = 04be734c-0d41-11e2-a7ed-001a4a013f65, targetVdsId = 0b5eb534-08fc-11e2-a34d-001a4a013f65, action = Stop, ip = dell-pem1000e-01-bl7.mgmt.eng.brq.redhat.com, port = , type = ipmilan, user = root, password = ******, options = ''), log id: 65f30b8c
>2012-10-04 10:25:02,351 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (pool-4-thread-50) [46f55832] FINISH, FenceVdsVDSCommand, return: Done, log id: 65f30b8c
>2012-10-04 10:25:02,351 INFO  [org.ovirt.engine.core.bll.FenceVdsBaseCommand] (pool-4-thread-50) [46f55832] Waiting for vds slot-7 to stop
>2012-10-04 10:25:07,352 INFO  [org.ovirt.engine.core.bll.FenceVdsBaseCommand] (pool-4-thread-50) [46f55832] Attempt 1 to get vds slot-7 status
>2012-10-04 10:25:07,404 INFO  [org.ovirt.engine.core.bll.FencingExecutor] (pool-4-thread-50) [46f55832] Executing <Status> Power Management command, Proxy Host:srh-211, Agent:ipmilan, Target Host:slot-7, Management IP:dell-pem1000e-01-bl7.mgmt.eng.brq.redhat.com, User:root, Options:
>2012-10-04 10:25:07,405 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (pool-4-thread-50) [46f55832] START, FenceVdsVDSCommand(HostName = srh-211, HostId = 04be734c-0d41-11e2-a7ed-001a4a013f65, targetVdsId = 0b5eb534-08fc-11e2-a34d-001a4a013f65, action = Status, ip = dell-pem1000e-01-bl7.mgmt.eng.brq.redhat.com, port = , type = ipmilan, user = root, password = ******, options = ''), log id: 26965a76
>2012-10-04 10:25:07,490 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (pool-4-thread-50) [46f55832] FINISH, FenceVdsVDSCommand, return: Test Succeeded, Host Status is: off, log id: 26965a76
>2012-10-04 10:25:07,490 INFO  [org.ovirt.engine.core.bll.FenceVdsBaseCommand] (pool-4-thread-50) [46f55832] vds slot-7 status is off
>2012-10-04 10:25:07,490 INFO  [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-4-thread-50) [46f55832] START, SetVmStatusVDSCommand( vmId = 2ad8d0e2-944e-406f-9ac0-8dd370e9aefb, status = Down), log id: 590d0cc4
>2012-10-04 10:25:07,498 INFO  [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (pool-4-thread-50) [46f55832] FINISH, SetVmStatusVDSCommand, log id: 590d0cc4
>2012-10-04 10:25:07,529 INFO  [org.ovirt.engine.core.bll.RunVmCommand] (pool-4-thread-50) [46f55832] Lock Acquired to object EngineLock [exclusiveLocks= key: 2ad8d0e2-944e-406f-9ac0-8dd370e9aefb value: VM
>, sharedLocks= ]
>2012-10-04 10:25:07,533 INFO  [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (pool-4-thread-50) [46f55832] START, IsVmDuringInitiatingVDSCommand( vmId = 2ad8d0e2-944e-406f-9ac0-8dd370e9aefb), log id: 460909ec
>2012-10-04 10:25:07,533 INFO  [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (pool-4-thread-50) [46f55832] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 460909ec
>2012-10-04 10:25:07,571 INFO  [org.ovirt.engine.core.bll.RunVmCommand] (pool-4-thread-44) [1b3fa77] Running command: RunVmCommand internal: true. Entities affected :  ID: 2ad8d0e2-944e-406f-9ac0-8dd370e9aefb Type: VM
>2012-10-04 10:25:07,577 INFO  [org.ovirt.engine.core.vdsbroker.UpdateVdsVMsClearedVDSCommand] (pool-4-thread-50) [46f55832] START, UpdateVdsVMsClearedVDSCommand(HostName = slot-7, HostId = 0b5eb534-08fc-11e2-a34d-001a4a013f65), log id: 7b4aee48
>2012-10-04 10:25:07,579 INFO  [org.ovirt.engine.core.vdsbroker.UpdateVdsVMsClearedVDSCommand] (pool-4-thread-50) [46f55832] FINISH, UpdateVdsVMsClearedVDSCommand, log id: 7b4aee48
>2012-10-04 10:25:07,595 INFO  [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (pool-4-thread-50) [6e941f27] Lock Acquired to object EngineLock [exclusiveLocks= key: 0b5eb534-08fc-11e2-a34d-001a4a013f65 value: VDS_FENCE
>, sharedLocks= ]
>2012-10-04 10:25:07,595 INFO  [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (pool-4-thread-50) [6e941f27] Running command: FenceVdsManualyCommand internal: true. Entities affected :  ID: 0b5eb534-08fc-11e2-a34d-001a4a013f65 Type: VDS
>2012-10-04 10:25:07,596 INFO  [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (pool-4-thread-50) [6e941f27] Lock freed to object EngineLock [exclusiveLocks= key: 0b5eb534-08fc-11e2-a34d-001a4a013f65 value: VDS_FENCE
>, sharedLocks= ]
>2012-10-04 10:25:07,627 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IsoPrefixVDSCommand] (pool-4-thread-44) [1b3fa77] START, IsoPrefixVDSCommand( storagePoolId = 40512d26-08a2-11e2-8a0b-001a4a013f65, ignoreFailoverLimit = false, compatabilityVersion = null), log id: ebf3eb2
>2012-10-04 10:25:07,627 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IsoPrefixVDSCommand] (pool-4-thread-44) [1b3fa77] FINISH, IsoPrefixVDSCommand, return: /rhev/data-center/40512d26-08a2-11e2-8a0b-001a4a013f65/28b4db25-9ea3-42f5-a5bf-0673751d56cf/images/11111111-1111-1111-1111-111111111111, log id: ebf3eb2
>2012-10-04 10:25:07,638 INFO  [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-4-thread-44) [1b3fa77] START, CreateVmVDSCommand(HostName = srh-206, HostId = 22855b2c-0c9a-11e2-b965-001a4a013f65, vmId=2ad8d0e2-944e-406f-9ac0-8dd370e9aefb, vm=org.ovirt.engine.core.common.businessentities.VM@2dc680d), log id: 2dad1fae
>2012-10-04 10:25:07,646 INFO  [org.ovirt.engine.core.bll.FencingExecutor] (pool-4-thread-50) Executing <Status> Power Management command, Proxy Host:srh-211, Agent:ipmilan, Target Host:slot-7, Management IP:dell-pem1000e-01-bl7.mgmt.eng.brq.redhat.com, User:root, Options:
>2012-10-04 10:25:07,647 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (pool-4-thread-50) START, FenceVdsVDSCommand(HostName = srh-211, HostId = 04be734c-0d41-11e2-a7ed-001a4a013f65, targetVdsId = 0b5eb534-08fc-11e2-a34d-001a4a013f65, action = Status, ip = dell-pem1000e-01-bl7.mgmt.eng.brq.redhat.com, port = , type = ipmilan, user = root, password = ******, options = ''), log id: 40f36bd4
>2012-10-04 10:25:07,653 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-4-thread-44) [1b3fa77] START, CreateVDSCommand(HostName = srh-206, HostId = 22855b2c-0c9a-11e2-b965-001a4a013f65, vmId=2ad8d0e2-944e-406f-9ac0-8dd370e9aefb, vm=org.ovirt.engine.core.common.businessentities.VM@2dc680d), log id: 2d2c4c9c
>2012-10-04 10:25:07,715 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-4-thread-44) [1b3fa77] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand spiceSslCipherSuite=DEFAULT,memSize=1024,kvmEnable=true,smp=4,emulatedMachine=rhel6.3.0,vmType=kvm,keyboardLayout=en-us,pitReinjection=false,nice=0,display=qxl,smpCoresPerSocket=2,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,timeOffset=50402,transparentHugePages=true,vmId=2ad8d0e2-944e-406f-9ac0-8dd370e9aefb,devices=[Ljava.util.Map;@7391f231,acpiEnable=true,vmName=ha-high,cpuType=Penryn,custom={device_78b6416d-e0fa-4990-9be4-c787985f0257device_09c1598b-1d47-40ef-8690-069479295b36=VmDevice {vmId=2ad8d0e2-944e-406f-9ac0-8dd370e9aefb, deviceId=09c1598b-1d47-40ef-8690-069479295b36, device=virtio-serial, type=controller, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x04, function=0x0}, managed=false, plugged=true, readOnly=false, alias=virtio-serial0}, device_78b6416d-e0fa-4990-9be4-c787985f0257=VmDevice {vmId=2ad8d0e2-944e-406f-9ac0-8dd370e9aefb, deviceId=78b6416d-e0fa-4990-9be4-c787985f0257, device=ide, type=controller, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, alias=ide0}, device_78b6416d-e0fa-4990-9be4-c787985f0257device_09c1598b-1d47-40ef-8690-069479295b36device_dedf470e-ad4d-4cc4-ad5f-d3bad600aecc=VmDevice {vmId=2ad8d0e2-944e-406f-9ac0-8dd370e9aefb, deviceId=dedf470e-ad4d-4cc4-ad5f-d3bad600aecc, device=unix, type=channel, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, alias=channel0}, device_78b6416d-e0fa-4990-9be4-c787985f0257device_09c1598b-1d47-40ef-8690-069479295b36device_dedf470e-ad4d-4cc4-ad5f-d3bad600aeccdevice_fc3958a9-42e2-4a21-8608-2522e2c6be6fdevice_1c125bf1-00e3-43df-9a29-f98cfd0f6c73=VmDevice {vmId=2ad8d0e2-944e-406f-9ac0-8dd370e9aefb, deviceId=1c125bf1-00e3-43df-9a29-f98cfd0f6c73, device=spicevmc, type=channel, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, alias=channel2}, device_78b6416d-e0fa-4990-9be4-c787985f0257device_09c1598b-1d47-40ef-8690-069479295b36device_dedf470e-ad4d-4cc4-ad5f-d3bad600aeccdevice_fc3958a9-42e2-4a21-8608-2522e2c6be6f=VmDevice {vmId=2ad8d0e2-944e-406f-9ac0-8dd370e9aefb, deviceId=fc3958a9-42e2-4a21-8608-2522e2c6be6f, device=unix, type=channel, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, alias=channel1}}
>2012-10-04 10:25:07,715 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (pool-4-thread-44) [1b3fa77] FINISH, CreateVDSCommand, log id: 2d2c4c9c
>2012-10-04 10:25:07,715 INFO  [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-4-thread-44) [1b3fa77] IncreasePendingVms::CreateVmIncreasing vds srh-206 pending vcpu count, now 4. Vm: ha-high
>2012-10-04 10:25:07,721 INFO  [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (pool-4-thread-44) [1b3fa77] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 2dad1fae
>2012-10-04 10:25:07,721 INFO  [org.ovirt.engine.core.bll.RunVmCommand] (pool-4-thread-44) [1b3fa77] Lock freed to object EngineLock [exclusiveLocks= key: 2ad8d0e2-944e-406f-9ac0-8dd370e9aefb value: VM
>, sharedLocks= ]
>2012-10-04 10:25:07,728 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (pool-4-thread-50) FINISH, FenceVdsVDSCommand, return: Test Succeeded, Host Status is: off, log id: 40f36bd4
>2012-10-04 10:25:07,729 INFO  [org.ovirt.engine.core.bll.StartVdsCommand] (pool-4-thread-50) Running command: StartVdsCommand internal: true. Entities affected :  ID: 0b5eb534-08fc-11e2-a34d-001a4a013f65 Type: VDS
>2012-10-04 10:25:07,740 INFO  [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-50) START, SetVdsStatusVDSCommand(HostName = slot-7, HostId = 0b5eb534-08fc-11e2-a34d-001a4a013f65, status=NonResponsive, nonOperationalReason=NONE), log id: 66278ba
>2012-10-04 10:25:07,751 INFO  [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (pool-4-thread-50) FINISH, SetVdsStatusVDSCommand, log id: 66278ba
>2012-10-04 10:25:07,783 INFO  [org.ovirt.engine.core.bll.FencingExecutor] (pool-4-thread-50) Executing <Start> Power Management command, Proxy Host:srh-206, Agent:ipmilan, Target Host:slot-7, Management IP:dell-pem1000e-01-bl7.mgmt.eng.brq.redhat.com, User:root, Options:
>2012-10-04 10:25:07,794 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (pool-4-thread-50) START, FenceVdsVDSCommand(HostName = srh-206, HostId = 22855b2c-0c9a-11e2-b965-001a4a013f65, targetVdsId = 0b5eb534-08fc-11e2-a34d-001a4a013f65, action = Start, ip = dell-pem1000e-01-bl7.mgmt.eng.brq.redhat.com, port = , type = ipmilan, user = root, password = ******, options = ''), log id: 2c3add16
>2012-10-04 10:25:07,856 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (pool-4-thread-50) FINISH, FenceVdsVDSCommand, return: Done, log id: 2c3add16
>2012-10-04 10:25:07,856 INFO  [org.ovirt.engine.core.bll.FenceVdsBaseCommand] (pool-4-thread-50) Waiting for vds slot-7 to start
>2012-10-04 10:25:08,155 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (QuartzScheduler_Worker-9) START, FullListVdsCommand( HostId = 22855b2c-0c9a-11e2-b965-001a4a013f65, vds=null, vmIds=[2ad8d0e2-944e-406f-9ac0-8dd370e9aefb]), log id: 5aa153d4
>2012-10-04 10:25:08,170 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (QuartzScheduler_Worker-9) FINISH, FullListVdsCommand, return: [Lorg.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcStruct;@1586b1c8, log id: 5aa153d4
>2012-10-04 10:25:08,176 INFO  [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (QuartzScheduler_Worker-9) Recieved a memballoon Device without an address when processing VM 2ad8d0e2-944e-406f-9ac0-8dd370e9aefb devices, skipping device: {specParams={model=virtio}, device=memballoon, type=balloon, deviceId=34f4fb0a-e9ee-419b-a785-579a406b43fc}
>2012-10-04 10:25:10,248 INFO  [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (QuartzScheduler_Worker-48) [24bcd489] VM ha-high 2ad8d0e2-944e-406f-9ac0-8dd370e9aefb moved from WaitForLaunch --> PoweringUp
>2012-10-04 10:25:10,248 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (QuartzScheduler_Worker-48) [24bcd489] START, FullListVdsCommand( HostId = 22855b2c-0c9a-11e2-b965-001a4a013f65, vds=null, vmIds=[2ad8d0e2-944e-406f-9ac0-8dd370e9aefb]), log id: 7fa1105b
>2012-10-04 10:25:10,265 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (QuartzScheduler_Worker-48) [24bcd489] FINISH, FullListVdsCommand, return: [Lorg.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcStruct;@273907d3, log id: 7fa1105b
>2012-10-04 10:25:12,857 INFO  [org.ovirt.engine.core.bll.FenceVdsBaseCommand] (pool-4-thread-50) Attempt 1 to get vds slot-7 status
>2012-10-04 10:25:12,915 INFO  [org.ovirt.engine.core.bll.FencingExecutor] (pool-4-thread-50) Executing <Status> Power Management command, Proxy Host:srh-211, Agent:ipmilan, Target Host:slot-7, Management IP:dell-pem1000e-01-bl7.mgmt.eng.brq.redhat.com, User:root, Options:
>2012-10-04 10:25:12,917 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (pool-4-thread-50) START, FenceVdsVDSCommand(HostName = srh-211, HostId = 04be734c-0d41-11e2-a7ed-001a4a013f65, targetVdsId = 0b5eb534-08fc-11e2-a34d-001a4a013f65, action = Status, ip = dell-pem1000e-01-bl7.mgmt.eng.brq.redhat.com, port = , type = ipmilan, user = root, password = ******, options = ''), log id: ed076bc
>2012-10-04 10:25:13,016 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (pool-4-thread-50) FINISH, FenceVdsVDSCommand, return: Test Succeeded, Host Status is: on, log id: ed076bc
>2012-10-04 10:25:13,016 INFO  [org.ovirt.engine.core.bll.FenceVdsBaseCommand] (pool-4-thread-50) vds slot-7 status is on
>2012-10-04 10:25:47,821 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-53) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:25:52,854 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-23) [2efef06d] XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:25:55,855 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-80) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:26:00,888 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-92) [2caa3fe1] XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:26:03,888 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-68) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:26:06,577 INFO  [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (QuartzScheduler_Worker-43) VM ha-high 2ad8d0e2-944e-406f-9ac0-8dd370e9aefb moved from PoweringUp --> Up
>2012-10-04 10:26:06,617 INFO  [org.ovirt.engine.core.vdsbroker.UpdateVdsDynamicDataVDSCommand] (QuartzScheduler_Worker-43) START, UpdateVdsDynamicDataVDSCommand(HostName = srh-206, HostId = 22855b2c-0c9a-11e2-b965-001a4a013f65, vdsDynamic=org.ovirt.engine.core.common.businessentities.VdsDynamic@410bbb5b), log id: 295a4f32
>2012-10-04 10:26:06,620 INFO  [org.ovirt.engine.core.vdsbroker.UpdateVdsDynamicDataVDSCommand] (QuartzScheduler_Worker-43) FINISH, UpdateVdsDynamicDataVDSCommand, log id: 295a4f32
>2012-10-04 10:26:08,921 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-94) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:26:11,921 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-79) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:26:16,952 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-50) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:26:19,952 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-72) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:26:24,985 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-57) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:26:27,986 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-28) [67eb6774] XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:26:33,017 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-34) [6178b7dd] XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:26:36,018 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-29) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:26:41,050 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-3) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:26:44,050 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-54) [4a718276] XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:26:49,083 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-78) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:26:52,083 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-24) [3acf8fe2] XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:26:57,115 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-53) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:27:17,148 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-84) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:27:22,179 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-19) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:27:25,179 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-15) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:27:30,217 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-50) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:27:33,218 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-46) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:27:38,247 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-33) [488f6f85] XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:27:41,249 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-13) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:27:46,280 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-34) [6178b7dd] XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:27:49,281 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-35) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:27:54,315 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-82) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:27:57,316 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-67) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:28:02,346 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-73) [5b981995] XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:28:05,347 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-70) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:28:10,379 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-53) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:28:13,379 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-87) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:28:18,410 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-97) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:28:21,410 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-92) [2caa3fe1] XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:28:26,441 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-84) XML RPC error in command GetCapabilitiesVDS ( HostName = slot-7 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, NoRouteToHostException: No route to host 
>2012-10-04 10:28:51,768 INFO  [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (QuartzScheduler_Worker-66) [325bcca8] Running command: InitVdsOnUpCommand internal: true.
>2012-10-04 10:28:51,894 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (QuartzScheduler_Worker-66) [58dd1dd8] START, ValidateStorageServerConnectionVDSCommand(HostName = slot-7, HostId = 0b5eb534-08fc-11e2-a34d-001a4a013f65, storagePoolId = 40512d26-08a2-11e2-8a0b-001a4a013f65, storageType = NFS, connectionList = [{ id: 6620568a-08a2-11e2-a7f2-001a4a013f65, connection: ps-ad2-set.rhev.lab.eng.brq.redhat.com:/var/lib/exports/iso3, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: 220cea3f-3920-4bbc-ba84-f36eb99efaca, connection: 10.34.63.202:/mnt/export/nfs/lv1/pstehlik/nfs01, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 5caffeda
>2012-10-04 10:28:51,927 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ValidateStorageServerConnectionVDSCommand] (QuartzScheduler_Worker-66) [58dd1dd8] FINISH, ValidateStorageServerConnectionVDSCommand, return: {6620568a-08a2-11e2-a7f2-001a4a013f65=0, 220cea3f-3920-4bbc-ba84-f36eb99efaca=0}, log id: 5caffeda
>2012-10-04 10:28:51,927 INFO  [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (QuartzScheduler_Worker-66) [58dd1dd8] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected :  ID: 40512d26-08a2-11e2-8a0b-001a4a013f65 Type: StoragePool
>2012-10-04 10:28:51,947 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (QuartzScheduler_Worker-66) [58dd1dd8] START, ConnectStorageServerVDSCommand(HostName = slot-7, HostId = 0b5eb534-08fc-11e2-a34d-001a4a013f65, storagePoolId = 40512d26-08a2-11e2-8a0b-001a4a013f65, storageType = NFS, connectionList = [{ id: 6620568a-08a2-11e2-a7f2-001a4a013f65, connection: ps-ad2-set.rhev.lab.eng.brq.redhat.com:/var/lib/exports/iso3, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: 220cea3f-3920-4bbc-ba84-f36eb99efaca, connection: 10.34.63.202:/mnt/export/nfs/lv1/pstehlik/nfs01, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 6e7a3a31
>2012-10-04 10:28:52,441 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (QuartzScheduler_Worker-66) [58dd1dd8] FINISH, ConnectStorageServerVDSCommand, return: {6620568a-08a2-11e2-a7f2-001a4a013f65=0, 220cea3f-3920-4bbc-ba84-f36eb99efaca=0}, log id: 6e7a3a31
>2012-10-04 10:28:52,442 INFO  [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (QuartzScheduler_Worker-66) [58dd1dd8] Host slot-7 storage connection was succeeded 
>2012-10-04 10:28:52,473 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-66) [58dd1dd8] START, ConnectStoragePoolVDSCommand(HostName = slot-7, HostId = 0b5eb534-08fc-11e2-a34d-001a4a013f65, storagePoolId = 40512d26-08a2-11e2-8a0b-001a4a013f65, vds_spm_id = 1, masterDomainId = 97d9bd45-2028-47d5-a1a2-b9c1d3bdb421, masterVersion = 1), log id: 661c0351
>2012-10-04 10:28:58,492 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-66) [58dd1dd8] FINISH, ConnectStoragePoolVDSCommand, log id: 661c0351
>2012-10-04 10:28:58,584 INFO  [org.ovirt.engine.core.bll.FencingExecutor] (QuartzScheduler_Worker-66) [58dd1dd8] Executing <Status> Power Management command, Proxy Host:srh-206, Agent:ipmilan, Target Host:slot-7, Management IP:dell-pem1000e-01-bl7.mgmt.eng.brq.redhat.com, User:root, Options:
>2012-10-04 10:28:58,604 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (QuartzScheduler_Worker-66) [58dd1dd8] START, FenceVdsVDSCommand(HostName = srh-206, HostId = 22855b2c-0c9a-11e2-b965-001a4a013f65, targetVdsId = 0b5eb534-08fc-11e2-a34d-001a4a013f65, action = Status, ip = dell-pem1000e-01-bl7.mgmt.eng.brq.redhat.com, port = , type = ipmilan, user = root, password = ******, options = ''), log id: 4d736b54
>2012-10-04 10:28:58,696 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (QuartzScheduler_Worker-66) [58dd1dd8] FINISH, FenceVdsVDSCommand, return: Test Succeeded, Host Status is: on, log id: 4d736b54
>2012-10-04 10:28:58,703 INFO  [org.ovirt.engine.core.bll.MultipleActionsRunner] (pool-4-thread-46) [58dd1dd8] MultipleActionsRunner of type MigrateVmToServer invoked with no actions
>2012-10-04 10:28:58,729 INFO  [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (QuartzScheduler_Worker-66) [1e553edf] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected :  ID: 0b5eb534-08fc-11e2-a34d-001a4a013f65 Type: VDS
>2012-10-04 10:28:58,755 INFO  [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (QuartzScheduler_Worker-66) [7fca9961] Running command: HandleVdsVersionCommand internal: true. Entities affected :  ID: 0b5eb534-08fc-11e2-a34d-001a4a013f65 Type: VDS
>2012-10-04 10:30:00,001 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-71) [31dc06a] Autorecovering hosts is disabled, skipping
>2012-10-04 10:30:00,001 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-71) [31dc06a] Autorecovering storage domains is disabled, skipping
>2012-10-04 10:35:00,000 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-100) Autorecovering hosts is disabled, skipping
>2012-10-04 10:35:00,000 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-100) Autorecovering storage domains is disabled, skipping
>2012-10-04 10:40:00,000 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-7) [78b3fc0c] Autorecovering hosts is disabled, skipping
>2012-10-04 10:40:00,001 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-7) [78b3fc0c] Autorecovering storage domains is disabled, skipping
>2012-10-04 10:42:47,423 INFO  [org.ovirt.engine.core.bll.DbUserCacheManager] (QuartzScheduler_Worker-75) DbUserCacheManager::refreshAllUserData() - entered
>2012-10-04 10:42:47,443 ERROR [org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy] (QuartzScheduler_Worker-75) Authentication failed. The user is either locked or disabled
>2012-10-04 10:42:47,443 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (QuartzScheduler_Worker-75) Failed ldap search server LDAP://brq-ipa.rhev.lab.eng.brq.redhat.com:389 due to Authentication failed. The user is either locked or disabled. We should not try the next server
>2012-10-04 10:42:47,450 ERROR [org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy] (QuartzScheduler_Worker-75) Authentication failed. The user is either locked or disabled
>2012-10-04 10:42:47,450 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (QuartzScheduler_Worker-75) Failed ldap search server LDAP://dc-02.ad2.rhev.lab.eng.brq.redhat.com:389 due to Authentication failed. The user is either locked or disabled. We should not try the next server
>2012-10-04 10:45:00,001 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-79) Autorecovering hosts is disabled, skipping
>2012-10-04 10:45:00,001 INFO  [org.ovirt.engine.core.bll.AutoRecoveryManager] (QuartzScheduler_Worker-79) Autorecovering storage domains is disabled, skipping
>2012-10-04 10:45:44,926 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IsValidVDSCommand] (pool-4-thread-50) START, IsValidVDSCommand( storagePoolId = 40512d26-08a2-11e2-8a0b-001a4a013f65, ignoreFailoverLimit = false, compatabilityVersion = null), log id: 7374f9ab
>2012-10-04 10:45:44,926 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IsValidVDSCommand] (pool-4-thread-44) START, IsValidVDSCommand( storagePoolId = 40512d26-08a2-11e2-8a0b-001a4a013f65, ignoreFailoverLimit = false, compatabilityVersion = null), log id: 4b384c5
>2012-10-04 10:45:44,927 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IsValidVDSCommand] (pool-4-thread-50) FINISH, IsValidVDSCommand, return: true, log id: 7374f9ab
>2012-10-04 10:45:44,927 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.IsValidVDSCommand] (pool-4-thread-44) FINISH, IsValidVDSCommand, return: true, log id: 4b384c5
>2012-10-04 10:45:44,927 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.GetIsoListVDSCommand] (pool-4-thread-44) START, GetIsoListVDSCommand( storagePoolId = 40512d26-08a2-11e2-8a0b-001a4a013f65, ignoreFailoverLimit = false, compatabilityVersion = null), log id: 412eeab3
>2012-10-04 10:45:44,927 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.GetFloppyListVDSCommand] (pool-4-thread-50) START, GetFloppyListVDSCommand( storagePoolId = 40512d26-08a2-11e2-8a0b-001a4a013f65, ignoreFailoverLimit = false, compatabilityVersion = null), log id: 5daaeeda
>2012-10-04 10:45:44,929 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (pool-4-thread-44) Failed to decrypt Data must not be longer than 256 bytes
>2012-10-04 10:45:44,929 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.HsmGetIsoListVDSCommand] (pool-4-thread-44) START, HsmGetIsoListVDSCommand(HostName = srh-206, HostId = 22855b2c-0c9a-11e2-b965-001a4a013f65, storagePoolId=40512d26-08a2-11e2-8a0b-001a4a013f65), log id: 132c5151
>2012-10-04 10:45:44,949 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-85) Failed to decrypt Data must start with zero
>2012-10-04 10:45:44,979 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.HsmGetIsoListVDSCommand] (pool-4-thread-44) FINISH, HsmGetIsoListVDSCommand, return: [virtio-win-1.5.3.iso, RHEV-toolsSetup_3.1_7.iso], log id: 132c5151
>2012-10-04 10:45:44,979 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.GetIsoListVDSCommand] (pool-4-thread-44) FINISH, GetIsoListVDSCommand, return: [virtio-win-1.5.3.iso, RHEV-toolsSetup_3.1_7.iso], log id: 412eeab3
>2012-10-04 10:45:44,986 INFO  [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (pool-4-thread-44) Finished automatic refresh process for ISO file type with success, for storage domain id 28b4db25-9ea3-42f5-a5bf-0673751d56cf.
>2012-10-04 10:45:45,014 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.GetFloppyListVDSCommand] (pool-4-thread-50) FINISH, GetFloppyListVDSCommand, return: [virtio-win-1.5.3.vfd], log id: 5daaeeda
>2012-10-04 10:45:45,017 INFO  [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (pool-4-thread-50) Finished automatic refresh process for Floppy file type with success, for storage domain id 28b4db25-9ea3-42f5-a5bf-0673751d56cf.

Actions: View

Attachments on bug 863211: 621747