Bug 1562093 - Importing OVA as clone more than 1 time failed with disk_id already exists
Summary: Importing OVA as clone more than 1 time failed with disk_id already exists
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Virt
Version: 4.2.2
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ovirt-4.3.0
: ---
Assignee: Ryan Barry
QA Contact: Nisim Simsolo
URL:
Whiteboard:
Depends On: 1656092
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-03-29 13:56 UTC by Nisim Simsolo
Modified: 2019-02-13 07:43 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: When re-cloning a VM via a VM OVA file, duplicate Image IDs and Disk IDs cause errors when attempting to recreate the image. Consequence: Fails to import the VM OVA file even though the VM name has been changed because the Identifiers already exist. Fix: Added the ability to regenerate the Identifiers, added image mapping to correlate the previous Image ID to the new Image ID when copying the image and moved the attach image handling so that it will no be called when creating a new image in the database fails. Result: Re-cloning VMs from VM ova files now succeeds.
Clone Of:
Environment:
Last Closed: 2019-02-13 07:43:15 UTC
oVirt Team: Virt
Embargoed:
rule-engine: ovirt-4.3+


Attachments (Terms of Use)
engine.log look at 2018-03-29 16:59:18,491+03 ERROR (1.21 MB, text/plain)
2018-03-29 14:01 UTC, Nisim Simsolo
no flags Details
vdsm.log (3.02 MB, text/plain)
2018-03-29 14:02 UTC, Nisim Simsolo
no flags Details
engine_log_import_vmware (415.16 KB, application/x-xz)
2018-07-17 11:40 UTC, Israel Pinto
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 91545 0 master MERGED core: honor vm serial number policy again 2018-05-24 13:02:19 UTC
oVirt gerrit 91546 0 master ABANDONED core: validation for import ova as clone 2018-11-04 01:56:21 UTC

Description Nisim Simsolo 2018-03-29 13:56:30 UTC
Description of problem:
When importing same OVA as clone more than 1 time, it failed with the next engine.log ERROR:

2018-03-29 14:53:18,914+03 ERROR [org.ovirt.engine.core.bll.exportimport.ImportVmFromOvaCommand] (EE-ManagedThreadFactory-engine-Thread-24000) [b6c476a1-ec25-475c-955c-4ce69e5
2014a] Command 'org.ovirt.engine.core.bll.exportimport.ImportVmFromOvaCommand' failed: org.springframework.dao.DuplicateKeyException: CallableStatementCallback; SQL [{call ins
ertdiskvmelement(?, ?, ?, ?, ?, ?)}]; ERROR: duplicate key value violates unique constraint "pk_disk_vm_element"

This issue does not occur when importing external provider VM as clone.

Version-Release number of selected component (if applicable):
rhvm-4.2.2.5-0.1.el7
libvirt-client-3.9.0-14.el7_5.2.x86_64
qemu-kvm-rhev-2.10.0-21.el7_5.1.x86_64
sanlock-3.6.0-1.el7.x86_64
vdsm-4.20.23-1.el7ev.x86_64
virt-v2v-1.36.10-6.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. Import OVA. 
2. Import same OVA as clone.
3. Import again same OVA as clone.

Actual results:
Import failed due to volume duplication.

Expected results:
Import should succeed.

Additional info:
engine.log and vdsm.log attached

Comment 1 Nisim Simsolo 2018-03-29 14:01:20 UTC
Created attachment 1414781 [details]
engine.log look at 2018-03-29 16:59:18,491+03 ERROR

Comment 2 Nisim Simsolo 2018-03-29 14:02:44 UTC
Created attachment 1414783 [details]
vdsm.log

Comment 3 Liran Rotenberg 2018-05-16 11:16:19 UTC
Additional info:
The import process of the OVA again causing to the first VM disk to change status from OK to Illegal. 
Raising the severity of this bug.

Comment 4 Arik 2018-05-17 15:21:06 UTC
Supporting import of the same OVA more than once (i.e., as a clone) is a low priority.
However, We should avoid the exception and the transition of the disks to an illegal state. This can be achieved by adding a proper validation.

Comment 6 Israel Pinto 2018-07-17 11:38:13 UTC
We saw it also on V2V import from vmware, after problem with storage 
we get the duplicate exception.
from engine log:
2018-07-16 16:14:19,455+03 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-13) [503be901] EVENT_ID: ADD_DISK_INTERNAL_FAILURE(2,037), Add-Disk operation of 'test_v2v_windows_2012_automation_vmware' failed to complete.
2018-07-16 16:14:19,563+03 ERROR [org.ovirt.engine.core.bll.exportimport.ImportVmFromExternalProviderCommand] (default task-13) [503be901] Command 'org.ovirt.engine.core.bll.exportimport.ImportVmFromExternalProviderCommand' failed: org.springframework.dao.DuplicateKeyException: CallableStatementCallback; SQL [{call insertdiskvmelement(?, ?, ?, ?, ?, ?)}]; ERROR: duplicate key value violates unique constraint "pk_disk_vm_element"
  Detail: Key (vm_id, disk_id)=(003eaa6f-590c-4d0d-9a4b-457701691c6d, a03eb673-da26-4ba3-a218-149df795271e) already exists.
  Where: SQL statement "INSERT INTO disk_vm_element (
        disk_id,
        vm_id,
        is_boot,
        pass_discard,
        disk_interface,
        is_using_scsi_reservation)
    VALUES (
        v_disk_id,
        v_vm_id,
        v_is_boot,
        v_pass_discard,
        v_disk_interface,
        v_is_using_scsi_reservation)"
PL/pgSQL function insertdiskvmelement(uuid,uuid,boolean,boolean,character varying,boolean) line 3 at SQL statement; nested exception is org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "pk_disk_vm_element"
  Detail: Key (vm_id, disk_id)=(003eaa6f-590c-4d0d-9a4b-457701691c6d, a03eb673-da26-4ba3-a218-149df795271e) already exists.
  Where: SQL statement "INSERT INTO disk_vm_element (
        disk_id,
        vm_id,
        is_boot,
        pass_discard,
        disk_interface,
        is_using_scsi_reservation)
    VALUES (
        v_disk_id,
        v_vm_id,
        v_is_boot,
        v_pass_discard,
        v_disk_interface,
        v_is_using_scsi_reservation)"
PL/pgSQL function insertdiskvmelement(uuid,uuid,boolean,boolean,character varying,boolean) line 3 at SQL statement
2018-07-16 16:14:19,563+03 ERROR [org.ovirt.engine.core.bll.exportimport.ImportVmFromExternalProviderCommand] (default task-13) [503be901] Exception: javax.ejb.EJBException: org.springframework.dao.DuplicateKeyException: CallableStatementCallback; SQL [{call insertdiskvmelement(?, ?, ?, ?, ?, ?)}]; ERROR: duplicate key value violates unique constraint "pk_disk_vm_element"
  Detail: Key (vm_id, disk_id)=(003eaa6f-590c-4d0d-9a4b-457701691c6d, a03eb673-da26-4ba3-a218-149df795271e) already exists.
  Where: SQL statement "INSERT INTO disk_vm_element (
        disk_id,
        vm_id,
        is_boot,
        pass_discard,
        disk_interface,
        is_using_scsi_reservation)
    VALUES (
        v_disk_id,
        v_vm_id,
        v_is_boot,
        v_pass_discard,
        v_disk_interface,
        v_is_using_scsi_reservation)"
PL/pgSQL function insertdiskvmelement(uuid,uuid,boolean,boolean,character varying,boolean) line 3 at SQL statement; nested exception is org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "pk_disk_vm_element"
  Detail: Key (vm_id, disk_id)=(003eaa6f-590c-4d0d-9a4b-457701691c6d, a03eb673-da26-4ba3-a218-149df795271e) already exists.
  Where: SQL statement "INSERT INTO disk_vm_element (
        disk_id,
        vm_id,
        is_boot,
        pass_discard,
        disk_interface,
        is_using_scsi_reservation)
    VALUES (
        v_disk_id,
        v_vm_id,
        v_is_boot,
        v_pass_discard,
        v_disk_interface,
        v_is_using_scsi_reservation)"
PL/pgSQL function insertdiskvmelement(uuid,uuid,boolean,boolean,character varying,boolean) line 3 at SQL statement

Adding engine log.

Comment 7 Israel Pinto 2018-07-17 11:40:45 UTC
Created attachment 1459404 [details]
engine_log_import_vmware

Comment 9 Arik 2018-08-06 13:03:45 UTC
(In reply to Israel Pinto from comment #6)
I think that deserves a separate bug - if the cleanup on failure is incomplete and thus leads to DuplicateKeyException on the following attempt, it is a different issue than supporting import-as-clone.

Comment 10 Arik 2018-08-06 13:05:19 UTC
(In reply to Arik from comment #4)
> Supporting import of the same OVA more than once (i.e., as a clone) is a low
> priority.
> However, We should avoid the exception and the transition of the disks to an
> illegal state. This can be achieved by adding a proper validation.

Most of the work needed to support import-as-clone was done as part of bz 1526033. Adjusting import-vm-from-ova to that should not be too difficult.

Comment 11 Nisim Simsolo 2018-11-19 09:50:57 UTC
(In reply to Arik from comment #9)
> (In reply to Israel Pinto from comment #6)
> I think that deserves a separate bug - if the cleanup on failure is
> incomplete and thus leads to DuplicateKeyException on the following attempt,
> it is a different issue than supporting import-as-clone.

Reproduced. Please see https://bugzilla.redhat.com/show_bug.cgi?id=1651137

Comment 12 Arik 2018-12-24 17:19:25 UTC
Ryan, this bug looks like similar to the one Steven is working on now, but triggered from a bit different scenario.
Could you please mark that somehow not as a duplicate but as a bug that should be verified once the fix for the bug Steven works on is resolved?

Comment 13 Steven Rosenberg 2018-12-27 15:44:51 UTC
The fix for "https://bugzilla.redhat.com/show_bug.cgi?id=1656092" does resolve this issue as per development testing.

Comment 14 Nisim Simsolo 2019-01-31 09:55:40 UTC
Verified:
ovirt-engine-4.3.0-0.8.master.20190122121624.git9a8a519.el7
vdsm-4.30.8-9.gitf28e377.el7.x86_64
qemu-kvm-ev-2.12.0-18.el7_6.3.1.x86_64
libvirt-client-4.5.0-10.el7_6.4.x86_64
virt-v2v-1.38.2-12.el7_6.1.x86_64

Comment 15 Sandro Bonazzola 2019-02-13 07:43:15 UTC
This bugzilla is included in oVirt 4.3.0 release, published on February 4th 2019.

Since the problem described in this bug report should be
resolved in oVirt 4.3.0 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.