Descriptionvvyazmin@redhat.com
2013-07-23 12:32:55 UTC
Created attachment 777302[details]
## Logs rhevm, vdsm, libvirt, thread dump, superVdsm
Description of problem:
Failed LSM on FCP Storage Domain Failed in VmReplicateDiskStartVDS
Version-Release number of selected component (if applicable):
RHEVM 3.3 - IS6 environment:
RHEVM: rhevm-3.3.0-0.9.master.el6ev.noarch
VDSM: vdsm-4.12.0-rc1.12.git8ee6885.el6.x86_64
LIBVIRT: libvirt-0.10.2-18.el6_4.9.x86_64
QEMU & KVM: qemu-kvm-rhev-0.12.1.2-2.355.el6_4.5.x86_64
SANLOCK: sanlock-2.6-2.el6.x86_64
How reproducible:
100%
Steps to Reproduce:
Create FCP Data Center with two hosts connected to two Storage Domains.
Create VM with one disk
Run a VM with Run-Once and “Start in Pause Mode” selected on the HSM server.
Try run Live Storage Migration (LVM)
Actual results:
Failed run LVM
Expected results:
Succeed run LVM
Impact on user:
Workaround:
none
Additional info:
/var/log/ovirt-engine/engine.log
2013-07-23 11:50:47,018 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskStartVDSCommand] (pool-5-thread-123) [17aa3b6] START, VmReplicateDiskStartVDSCommand(HostName = tigris02.scl.lab.tlv.redhat.com, HostId = 3cca5914-a9
84-45e5-9e02-1e22f2073049, vmId=0f91a910-8758-44dd-b3e6-5888f1c064e0), log id: 504979cb
2013-07-23 11:50:47,036 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskStartVDSCommand] (pool-5-thread-123) [17aa3b6] Failed in VmReplicateDiskStartVDS method
2013-07-23 11:50:47,036 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskStartVDSCommand] (pool-5-thread-123) [17aa3b6] Error code imageErr and error message VDSGenericException: VDSErrorException: Failed to VmReplicateDis
kStartVDS, error = Drive image file %s could not be found
2013-07-23 11:50:47,036 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskStartVDSCommand] (pool-5-thread-123) [17aa3b6] Command org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskStartVDSCommand return value
StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=13, mMessage=Drive image file %s could not be found]]
2013-07-23 11:50:47,036 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskStartVDSCommand] (pool-5-thread-123) [17aa3b6] HostName = tigris02.scl.lab.tlv.redhat.com
2013-07-23 11:50:47,036 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskStartVDSCommand] (pool-5-thread-123) [17aa3b6] Command VmReplicateDiskStartVDS execution failed. Exception: VDSErrorException: VDSGenericException: V
DSErrorException: Failed to VmReplicateDiskStartVDS, error = Drive image file %s could not be found
2013-07-23 11:50:47,036 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskStartVDSCommand] (pool-5-thread-123) [17aa3b6] FINISH, VmReplicateDiskStartVDSCommand, log id: 504979cb
2013-07-23 11:50:47,036 ERROR [org.ovirt.engine.core.bll.lsm.VmReplicateDiskStartTaskHandler] (pool-5-thread-123) [17aa3b6] Failed VmReplicateDiskStart (Disk 506c4e21-0da2-4bee-b0a3-8813645f6eac , VM 0f91a910-8758-44dd-b3e6-5888f1c064e0)
2013-07-23 11:50:47,036 ERROR [org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand] (pool-5-thread-123) [17aa3b6] Command org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand throw Vdc Bll exception. With error message VdcBLLException:
Drive image file imageErr could not be found (Failed with VDSM error imageErr and code 13)
2013-07-23 11:50:47,038 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand] (pool-5-thread-123) [17aa3b6] START, VmReplicateDiskFinishVDSCommand(HostName = tigris02.scl.lab.tlv.redhat.com, HostId = 3cca5914-
a984-45e5-9e02-1e22f2073049, vmId=0f91a910-8758-44dd-b3e6-5888f1c064e0), log id: 5d1d5a05
2013-07-23 11:50:47,044 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand] (pool-5-thread-123) [17aa3b6] Failed in VmReplicateDiskFinishVDS method
2013-07-23 11:50:47,044 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand] (pool-5-thread-123) [17aa3b6] Error code imageErr and error message VDSGenericException: VDSErrorException: Failed to VmReplicateDi
skFinishVDS, error = Drive image file %s could not be found
2013-07-23 11:50:47,044 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand] (pool-5-thread-123) [17aa3b6] Command org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand return value
StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=13, mMessage=Drive image file %s could not be found]]
2013-07-23 11:50:47,045 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand] (pool-5-thread-123) [17aa3b6] HostName = tigris02.scl.lab.tlv.redhat.com
2013-07-23 11:50:47,045 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand] (pool-5-thread-123) [17aa3b6] Command VmReplicateDiskFinishVDS execution failed. Exception: VDSErrorException: VDSGenericException:
VDSErrorException: Failed to VmReplicateDiskFinishVDS, error = Drive image file %s could not be found
/var/log/vdsm/vdsm.log
Comment 2vvyazmin@redhat.com
2013-07-23 13:27:06 UTC
Same problem found on iSCSI Data Center environment
Comment 3vvyazmin@redhat.com
2013-07-23 13:29:46 UTC
Created attachment 777302 [details] ## Logs rhevm, vdsm, libvirt, thread dump, superVdsm Description of problem: Failed LSM on FCP Storage Domain Failed in VmReplicateDiskStartVDS Version-Release number of selected component (if applicable): RHEVM 3.3 - IS6 environment: RHEVM: rhevm-3.3.0-0.9.master.el6ev.noarch VDSM: vdsm-4.12.0-rc1.12.git8ee6885.el6.x86_64 LIBVIRT: libvirt-0.10.2-18.el6_4.9.x86_64 QEMU & KVM: qemu-kvm-rhev-0.12.1.2-2.355.el6_4.5.x86_64 SANLOCK: sanlock-2.6-2.el6.x86_64 How reproducible: 100% Steps to Reproduce: Create FCP Data Center with two hosts connected to two Storage Domains. Create VM with one disk Run a VM with Run-Once and “Start in Pause Mode” selected on the HSM server. Try run Live Storage Migration (LVM) Actual results: Failed run LVM Expected results: Succeed run LVM Impact on user: Workaround: none Additional info: /var/log/ovirt-engine/engine.log 2013-07-23 11:50:47,018 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskStartVDSCommand] (pool-5-thread-123) [17aa3b6] START, VmReplicateDiskStartVDSCommand(HostName = tigris02.scl.lab.tlv.redhat.com, HostId = 3cca5914-a9 84-45e5-9e02-1e22f2073049, vmId=0f91a910-8758-44dd-b3e6-5888f1c064e0), log id: 504979cb 2013-07-23 11:50:47,036 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskStartVDSCommand] (pool-5-thread-123) [17aa3b6] Failed in VmReplicateDiskStartVDS method 2013-07-23 11:50:47,036 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskStartVDSCommand] (pool-5-thread-123) [17aa3b6] Error code imageErr and error message VDSGenericException: VDSErrorException: Failed to VmReplicateDis kStartVDS, error = Drive image file %s could not be found 2013-07-23 11:50:47,036 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskStartVDSCommand] (pool-5-thread-123) [17aa3b6] Command org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskStartVDSCommand return value StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=13, mMessage=Drive image file %s could not be found]] 2013-07-23 11:50:47,036 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskStartVDSCommand] (pool-5-thread-123) [17aa3b6] HostName = tigris02.scl.lab.tlv.redhat.com 2013-07-23 11:50:47,036 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskStartVDSCommand] (pool-5-thread-123) [17aa3b6] Command VmReplicateDiskStartVDS execution failed. Exception: VDSErrorException: VDSGenericException: V DSErrorException: Failed to VmReplicateDiskStartVDS, error = Drive image file %s could not be found 2013-07-23 11:50:47,036 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskStartVDSCommand] (pool-5-thread-123) [17aa3b6] FINISH, VmReplicateDiskStartVDSCommand, log id: 504979cb 2013-07-23 11:50:47,036 ERROR [org.ovirt.engine.core.bll.lsm.VmReplicateDiskStartTaskHandler] (pool-5-thread-123) [17aa3b6] Failed VmReplicateDiskStart (Disk 506c4e21-0da2-4bee-b0a3-8813645f6eac , VM 0f91a910-8758-44dd-b3e6-5888f1c064e0) 2013-07-23 11:50:47,036 ERROR [org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand] (pool-5-thread-123) [17aa3b6] Command org.ovirt.engine.core.bll.lsm.LiveMigrateDiskCommand throw Vdc Bll exception. With error message VdcBLLException: Drive image file imageErr could not be found (Failed with VDSM error imageErr and code 13) 2013-07-23 11:50:47,038 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand] (pool-5-thread-123) [17aa3b6] START, VmReplicateDiskFinishVDSCommand(HostName = tigris02.scl.lab.tlv.redhat.com, HostId = 3cca5914- a984-45e5-9e02-1e22f2073049, vmId=0f91a910-8758-44dd-b3e6-5888f1c064e0), log id: 5d1d5a05 2013-07-23 11:50:47,044 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand] (pool-5-thread-123) [17aa3b6] Failed in VmReplicateDiskFinishVDS method 2013-07-23 11:50:47,044 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand] (pool-5-thread-123) [17aa3b6] Error code imageErr and error message VDSGenericException: VDSErrorException: Failed to VmReplicateDi skFinishVDS, error = Drive image file %s could not be found 2013-07-23 11:50:47,044 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand] (pool-5-thread-123) [17aa3b6] Command org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand return value StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=13, mMessage=Drive image file %s could not be found]] 2013-07-23 11:50:47,045 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand] (pool-5-thread-123) [17aa3b6] HostName = tigris02.scl.lab.tlv.redhat.com 2013-07-23 11:50:47,045 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VmReplicateDiskFinishVDSCommand] (pool-5-thread-123) [17aa3b6] Command VmReplicateDiskFinishVDS execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to VmReplicateDiskFinishVDS, error = Drive image file %s could not be found /var/log/vdsm/vdsm.log