Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1430865 - ERROR: duplicate key value violates unique constraint "pk_unregistered_disks_to_vms"
ERROR: duplicate key value violates unique constraint "pk_unregistered_disks_...
Status: CLOSED ERRATA
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine (Show other bugs)
4.0.6
All Linux
unspecified Severity high
: ovirt-4.2.0
: 4.2.0
Assigned To: Maor
Raz Tamir
: ZStream
Depends On:
Blocks: 1446920
  Show dependency treegraph
 
Reported: 2017-03-09 13:15 EST by Sam Yangsao
Modified: 2018-05-15 13:42 EDT (History)
14 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Previously, if a snapshot of a disk attached to a virtual machine was deleted and the user tried to attach the storage domain containing this virtual machine before the OVF_STORE had been updated with the change, the attachment operation would fail. Because the OVF indicated the presence of a disk with a snapshot, this disk was fetched as a potential disk to register, even though it was already part of a virtual machine. In the current release, the disks are counted only once and the storage domain can be attached.
Story Points: ---
Clone Of:
: 1446920 (view as bug list)
Environment:
Last Closed: 2018-05-15 13:41:09 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Storage
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
ylavi: testing_plan_complete?


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 74280 master MERGED core: Use set for disk ids fetched from VM's OVF. 2017-03-20 08:06 EDT
oVirt gerrit 74320 ovirt-engine-4.1 MERGED core: Use set for disk ids fetched from VM's OVF. 2017-03-20 09:20 EDT
Red Hat Product Errata RHEA-2018:1488 None None None 2018-05-15 13:42 EDT

  None (edit)
Description Sam Yangsao 2017-03-09 13:15:35 EST
Description of problem:

Unable to attach a data storage domain that was detached/removed

Version-Release number of selected component (if applicable):

4.0.6.3-0.1.el7ev

How reproducible:

Not always

Steps to Reproduce:

Unsure - while detaching the data storage domain, the OVS network went offline in another data center while this was occuring

Actual results:

Adding the storage domain back and attaching fails with the following errors:

[....]
2017-03-09 11:47:59,861 ERROR [org.ovirt.engine.core.bll.storage.disk.image.RegisterDiskCommand] (org.ovirt.thread.pool-6-thread-33) [2ffff1f4] Transaction rolled-back for command 'org.ovirt.engine.core.bll.storage.disk.image.RegisterDiskCommand'.
2017-03-09 11:47:59,861 ERROR [org.ovirt.engine.core.bll.storage.disk.image.RegisterDiskCommand] (org.ovirt.thread.pool-6-thread-33) [2ffff1f4] Transaction rolled-back for command 'org.ovirt.engine.core.bll.storage.disk.image.RegisterDiskCommand'.
2017-03-09 11:47:59,861 INFO  [org.ovirt.engine.core.utils.transaction.TransactionSupport] (org.ovirt.thread.pool-6-thread-33) [2ffff1f4] transaction rolled back
2017-03-09 11:47:59,862 ERROR [org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand] (org.ovirt.thread.pool-6-thread-33) [2ffff1f4] Command 'org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand' failed: CallableStatementCallback; SQL [{call insertunregistereddiskstovms(?, ?, ?, ?)}]; ERROR: duplicate key value violates unique constraint "pk_unregistered_disks_to_vms"
  Detail: Key (disk_id, entity_id, storage_domain_id)=(f1f4cd1e-6bc7-4915-991c-49cb0abb9aef, a5e20a64-4160-4963-8dc0-68b86410c910, 9b161463-ce93-45e7-8f14-074d670dd32b) already exists.
  Where: SQL statement "INSERT INTO unregistered_disks_to_vms (
        disk_id,
        entity_id,
        entity_name,
        storage_domain_id
        )
    VALUES (
        v_disk_id,
        v_entity_id,
        v_entity_name,
        v_storage_domain_id
        )"
PL/pgSQL function insertunregistereddiskstovms(uuid,uuid,character varying,uuid) line 3 at SQL statement; nested exception is org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "pk_unregistered_disks_to_vms"
[....]

Expected results:

Attaching a storage domain back to its original location should work properly 

Additional info:

All engine.log output is located here:

http://pastebin.test.redhat.com/463201 - see lines 409 through 426 for the messages above.
Comment 4 Allon Mureinik 2017-03-12 10:28:23 EDT
This constraint was introduced in RHV 4.0.1 as part of the fix for bug 1302780. Maor - can you take a look please?
Comment 5 Maor 2017-03-12 11:35:29 EDT
Hi Sam,

Just wanted to clear something.
You wrote that you have one DC with 2 clusters, Dell-cluster and HP-cluster.
by clusters, did you mean linux clusters to get High Availability for RHEV-M?
Comment 6 Sam Yangsao 2017-03-13 11:42:37 EDT
(In reply to Maor from comment #5)
> Hi Sam,
> 
> Just wanted to clear something.
> You wrote that you have one DC with 2 clusters, Dell-cluster and HP-cluster.
> by clusters, did you mean linux clusters to get High Availability for RHEV-M?

Hey Maor,

It's 1 Data center with 2 clusters.  No HA for the RHV-M :)

Thanks!
Comment 10 Maor 2017-03-18 18:03:53 EDT
Hi,

I will take a look at it first think tomorrow morning.
Thank you for the info
Comment 13 Maor 2017-03-19 09:10:22 EDT
Hi Sam,

I think that I found the issue, thank you very much for your help and the access to your env, that was much helpful and reduced the time finding the issue.

It looks like there were VMs with disks and snapshots and some of the snapshots got deleted before there was an OVF update in the OVF_STORE disk.
In that point of time the OVF of the VM indicated the disks contains snapshots.
while those disks were without any snapshots.
Once the storage domain got attached those disks were fetched as potential disks to register which were part of the VMs also.

There seem to be a bug in the xml parser of the OVF that add those disks the VMs which those are attached to, since the XML was not updated after the removal of the snapshots those disks were initialized with VMs there were attached to, although those VMs were actually the same VM and that caused the SQL exception.

Steps to reproduce:
1. Create a VM with disks and snapshot
2. Delete the snapshot
3. force remove the storage domain (do not deactivate it since the OVF_STORE will be updated this way)
4. Try to attach the storage domain back again to the Data Center

I will post a patch that fixes it.
Thank you again for your help
Comment 14 Sam Yangsao 2017-03-20 09:49:50 EDT
Awe(In reply to Maor from comment #13)
> Hi Sam,
> 
> I think that I found the issue, thank you very much for your help and the
> access to your env, that was much helpful and reduced the time finding the
> issue.
> 
> It looks like there were VMs with disks and snapshots and some of the
> snapshots got deleted before there was an OVF update in the OVF_STORE disk.
> In that point of time the OVF of the VM indicated the disks contains
> snapshots.
> while those disks were without any snapshots.
> Once the storage domain got attached those disks were fetched as potential
> disks to register which were part of the VMs also.
> 
> There seem to be a bug in the xml parser of the OVF that add those disks the
> VMs which those are attached to, since the XML was not updated after the
> removal of the snapshots those disks were initialized with VMs there were
> attached to, although those VMs were actually the same VM and that caused
> the SQL exception.
> 
> Steps to reproduce:
> 1. Create a VM with disks and snapshot
> 2. Delete the snapshot
> 3. force remove the storage domain (do not deactivate it since the OVF_STORE
> will be updated this way)
> 4. Try to attach the storage domain back again to the Data Center
> 
> I will post a patch that fixes it.
> Thank you again for your help

Awesome, thanks for your hardwork in finding this Maor.
Comment 15 rhev-integ 2017-04-26 06:50:22 EDT
WARN: Bug status wasn't changed from MODIFIED to ON_QA due to the following reason:

[FOUND CLONE FLAGS: ['rhevm-4.1.z', 'rhevm-4.2-ga'], ]

For more info please contact: rhv-devops@redhat.com
Comment 17 Raz Tamir 2017-05-23 18:44:51 EDT
Verified with our automation on ovirt-4.2.0-0.0.master.20170519193842.gitf4353fb6.el7.centos.

No failures on importing and attaching storage domain with unregistered disks
Comment 20 errata-xmlrpc 2018-05-15 13:41:09 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:1488

Note You need to log in before you can comment on or make changes to this bug.