Description of problem: the auto-import of the hosted-engine VM from the engine fails with: 2016-12-20 20:23:04,905+02 INFO [org.ovirt.engine.core.bll.exportimport.ImportVmCommand] (org.ovirt.thread.pool-6-thread-18) [3143daa3] Lock Acquired to object 'EngineLock:{exclusiveLocks='[d74ea91a-8dc0-433e-a655-ffb6f018ce96=<VM, ACTION_TYPE_FAILED_VM_IS_BEING_IMPORTED$VmName HostedEngine>, HostedEngine=<VM_NAME, ACTION_TYPE_FAILED_NAME_ALREADY_USED>]', sharedLocks='[d74ea91a-8dc0-433e-a655-ffb6f018ce96=<REMOTE_VM, ACTION_TYPE_FAILED_VM_IS_BEING_IMPORTED$VmName HostedEngine>]'}' 2016-12-20 20:23:04,910+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-6-thread-18) [3143daa3] START, GetImageInfoVDSCommand( GetImageInfoVDSCommandParameters:{runAsync='true', storagePoolId='00000001-0001-0001-0001-000000000311', ignoreFailoverLimit='false', storageDomainId='73de876d-1138-4d56-a930-85f5195357b9', imageGroupId='ffa67066-8c85-4392-9e03-57ee80567c58', imageId='9d48f64e-8638-40bc-ad98-e26386b70c7d'}), log id: 7617c4d9 2016-12-20 20:23:04,911+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVolumeInfoVDSCommand] (org.ovirt.thread.pool-6-thread-18) [3143daa3] START, GetVolumeInfoVDSCommand(HostName = puma18.scl.lab.tlv.redhat.com, GetVolumeInfoVDSCommandParameters:{runAsync='true', hostId='24a83841-9862-447c-a5a5-f301fc86b8cb', storagePoolId='00000001-0001-0001-0001-000000000311', storageDomainId='73de876d-1138-4d56-a930-85f5195357b9', imageGroupId='ffa67066-8c85-4392-9e03-57ee80567c58', imageId='9d48f64e-8638-40bc-ad98-e26386b70c7d'}), log id: 12502694 2016-12-20 20:23:06,964+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVolumeInfoVDSCommand] (org.ovirt.thread.pool-6-thread-18) [3143daa3] Failed building DiskImage: java.lang.String cannot be cast to java.util.Map 2016-12-20 20:23:06,964+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVolumeInfoVDSCommand] (org.ovirt.thread.pool-6-thread-18) [3143daa3] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.GetVolumeInfoVDSCommand' return value ' VolumeInfoReturn:{status='Status [code=0, message=Done]'} status = OK domain = 73de876d-1138-4d56-a930-85f5195357b9 voltype = LEAF description = Hosted Engine Image parent = 00000000-0000-0000-0000-000000000000 format = RAW generation = 0 image = ffa67066-8c85-4392-9e03-57ee80567c58 ctime = 1482255474 disktype = 2 legality = LEGAL mtime = 0 apparentsize = 53687091200 children: [] pool = capacity = 53687091200 uuid = 9d48f64e-8638-40bc-ad98-e26386b70c7d truesize = 3719168000 type = SPARSE lease = EXCLUSIVE ' I think that the issue has been introduced here: https://gerrit.ovirt.org/#/c/68732 Version-Release number of selected component (if applicable): How reproducible: 100% Steps to Reproduce: 1. deploy hosted-engine 2. add the first regular storage domain 3. check the engine logs Actual results: Failed importing the Hosted Engine VM Expected results: sucesfully imported the Hosted Engine VM Additional info:
> I think that the issue has been introduced here: > https://gerrit.ovirt.org/#/c/68732 or here: https://gerrit.ovirt.org/#/c/67945/10/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/vdsbroker/GetVolumeInfoVDSCommand.java
Maor, according to Simone's analysis above, it seems as though one of your recent patches introduced this issue. Can you take a look please?
(In reply to Allon Mureinik from comment #2) > Maor, according to Simone's analysis above, it seems as though one of your > recent patches introduced this issue. Can you take a look please? Doesn't seem to be related, the patch only adds a null check. Looking into the exception I might guess it is related to an old VDSM version. commit 406a32bd2f8ff1a7fca9211e5c005ddef4cd2c79 has changed the return value of LEASE_FREE and sc.LEASE_EXCLUSIVE to a map. Simone, is the VDSM you are using includes this commit?
Nikolai hit this, look at https://bugzilla.redhat.com/show_bug.cgi?id=1398443#c12 He said he was using vdsm-4.18.999-1020.git1ff41b1.el7.centos.x86_64
(In reply to Simone Tiraboschi from comment #4) > Nikolai hit this, look at > https://bugzilla.redhat.com/show_bug.cgi?id=1398443#c12 I assume you meant https://bugzilla.redhat.com/show_bug.cgi?id=1398443#c14 > > He said he was using vdsm-4.18.999-1020.git1ff41b1.el7.centos.x86_64 Based on the build's git commit 1ff41b1, this looks like a build that was based on https://gerrit.ovirt.org/#/c/67619/ which was merged in Dec 1st although the patch which changed the lease type (https://gerrit.ovirt.org/#/c/67610/) was merged later in Dec 14th. can he test this with a newer VDSM build?
This bug report has Keywords: Regression or TestBlocker. Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.
Latest appliance build seams fine; moving to ON_QA, TestOnly
Initial logs not being reproduced. Works for me on these components on engine: Red Hat Virtualization Manager Version: 4.1.0.1-0.1.el7 rhev-guest-tools-iso-4.1-3.el7ev.noarch rhevm-doc-4.1.0-1.el7ev.noarch rhevm-dependencies-4.1.0-1.el7ev.noarch rhevm-setup-plugins-4.1.0-1.el7ev.noarch rhevm-4.1.0.1-0.1.el7.noarch rhevm-guest-agent-common-1.0.12-3.el7ev.noarch rhevm-branding-rhev-4.1.0-0.el7ev.noarch Linux version 3.10.0-514.6.1.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ) #1 SMP Sat Dec 10 11:15:38 EST 2016 Linux 3.10.0-514.6.1.el7.x86_64 #1 SMP Sat Dec 10 11:15:38 EST 2016 x86_64 x86_64 x86_64 GNU/Linux Red Hat Enterprise Linux Server release 7.3 (Maipo) On host: rhvm-appliance-4.1.20170119.1-1.el7ev.noarch ovirt-imageio-common-0.5.0-0.el7ev.noarch ovirt-host-deploy-1.6.0-1.el7ev.noarch ovirt-vmconsole-1.0.4-1.el7ev.noarch qemu-kvm-rhev-2.6.0-28.el7_3.3.x86_64 ovirt-vmconsole-host-1.0.4-1.el7ev.noarch rhev-release-4.0.6-1-001.noarch ovirt-engine-sdk-python-3.6.9.1-1.el7ev.noarch mom-0.5.8-1.el7ev.noarch vdsm-4.19.2-2.el7ev.x86_64 libvirt-client-2.0.0-10.el7_3.4.x86_64 sanlock-3.4.0-1.el7.x86_64 ovirt-hosted-engine-setup-2.1.0-2.el7ev.noarch ovirt-imageio-daemon-0.5.0-0.el7ev.noarch ovirt-hosted-engine-ha-2.1.0-1.el7ev.noarch ovirt-setup-lib-1.1.0-1.el7ev.noarch Linux version 3.10.0-514.6.1.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ) #1 SMP Sat Dec 10 11:15:38 EST 2016 Linux 3.10.0-514.6.1.el7.x86_64 #1 SMP Sat Dec 10 11:15:38 EST 2016 x86_64 x86_64 x86_64 GNU/Linux Red Hat Enterprise Linux Server release 7.3 (Maipo) 1-Deployed HE over iSCSI. 2-Added single NFS data storage domain. 3-Got hosted-storage auto-imported. Moving to verified.