Bug 1132568

Summary: unregistered VM should be the latest one related to the Storage Domain
Product: [Retired] oVirt Reporter: Ori Gofen <ogofen>
Component: ovirt-engine-coreAssignee: Maor <mlipchuk>
Status: CLOSED CURRENTRELEASE QA Contact: Ori Gofen <ogofen>
Severity: high Docs Contact:
Priority: unspecified    
Version: 3.5CC: acanan, amureini, bugs, ecohen, gklein, iheim, mlipchuk, rbalakri, tnisan, yeylon
Target Milestone: ---   
Target Release: 3.5.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: storage
Fixed In Version: ovirt-3.5.0_rc3 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-10-17 12:36:58 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
vdsm+engine logs none

Description Ori Gofen 2014-08-21 14:47:13 UTC
Created attachment 929225 [details]
vdsm+engine logs

Description of problem:
When attempting to remove a domain populated with templates and disks and while removing we restart the engine, it causes a PSQLException to be thrown.
and engine complains about data loss

2014-08-21 17:10:18,600 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-2) Correlation ID: 10754714, Call Stack: null, Custom Event ID: -1, Message: Failed to complete creation of Template t1 from VM <UNKNOWN>.
2014-08-21 17:10:18,603 ERROR [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-8-thread-2) [within thread]: endAction for action type AddVmTemplate threw an exception.: org.springframework.dao.DataIntegrityViolationException: CallableStatementCallback; SQL [{call deletevmtemplates(?)}]; ERROR: update or delete on table "vm_static" violates foreign key constraint "vm_templates_vm_static" on table "vm_static"
  Detail: Key (vm_guid)=(00000000-0000-0000-0000-000000000000) is still referenced from table "vm_static".
  Where: SQL statement "DELETE FROM vm_static WHERE vm_guid =  $1 "
PL/pgSQL function "deletevmtemplates" line 7 at SQL statement; nested exception is org.postgresql.util.PSQLException: ERROR: update or delete on table "vm_static" violates foreign key constraint "vm_templates_vm_static" on table "vm_static"
  Detail: Key (vm_guid)=(00000000-0000-0000-0000-000000000000) is still referenced from table "vm_static".
  Where: SQL statement "DELETE FROM vm_static WHERE vm_guid =  $1 "
PL/pgSQL function "deletevmtemplates" line 7 at SQL statement
        at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:245) [spring-jdbc.jar:3.1.1.RELEASE]
        at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1030) [spring-jdbc.jar:3.1.1.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.call(JdbcTemplate.java:1064) [spring-jdbc.jar:3.1.1.RELEASE]
        at org.springframework.jdbc.core.simple.AbstractJdbcCall.executeCallInternal(AbstractJdbcCall.java:388) [spring-jdbc.jar:3.1.1.RELEASE]
        at org.springframework.jdbc.core.simple.AbstractJdbcCall.doExecute(AbstractJdbcCall.java:351) [spring-jdbc.jar:3.1.1.RELEASE]
        at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE]
        at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:141) [dal.jar:]
        at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:136) [dal.jar:]
        at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeModification(SimpleJdbcCallsHandler.java:74) [dal.jar:]
        at org.ovirt.engine.core.dao.DefaultGenericDaoDbFacade.remove(DefaultGenericDaoDbFacade.java:107) [dal.jar:]
        at org.ovirt.engine.core.bll.AddVmTemplateCommand.endWithFailure(AddVmTemplateCommand.java:695) [bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.internalEndWithFailure(CommandBase.java:651) [bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.endActionInTransactionScope(CommandBase.java:559) [bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1876) [bll.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:210) [utils.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInRequired(TransactionSupport.java:149) [utils.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:118) [utils.jar:]
        at org.ovirt.engine.core.bll.CommandBase.endAction(CommandBase.java:487) [bll.jar:]
        at org.ovirt.engine.core.bll.tasks.DecoratedCommand.endAction(DecoratedCommand.java:17) [bll.jar:]
        at org.ovirt.engine.core.bll.tasks.CoCoAsyncTaskHelper.endAction(CoCoAsyncTaskHelper.java:317) [bll.jar:]
        at org.ovirt.engine.core.bll.tasks.CommandCoordinatorImpl.endAction(CommandCoordinatorImpl.java:350) [bll.jar:]
        at org.ovirt.engine.core.bll.tasks.CommandAsyncTask.EndCommandAction(CommandAsyncTask.java:149) [bll.jar:]
        at org.ovirt.engine.core.bll.tasks.CommandAsyncTask.access$000(CommandAsyncTask.java:28) [bll.jar:]
        at org.ovirt.engine.core.bll.tasks.CommandAsyncTask$1.run(CommandAsyncTask.java:109) [bll.jar:]
        at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65]



Version-Release number of selected component (if applicable):
rc1

How reproducible:
100%

Steps to Reproduce:
The steps I've made are too complicated probably could be reproduced in much easier way
1.create several domains(I had 5) with templates on them
2.detach and remove all domains
3.while removing restart oVirt-engine


Actual results:
PSQLException and probably data loss

Expected results:


Additional info:

Comment 1 Ori Gofen 2014-08-24 13:05:59 UTC
The bug is reproduced also when trying to attach a domain back to dc in a case that:
1.the domain contains an ovfstorefile which have data on an object that already exist in the system(it means that the object belongs to several domains)
2.the object is updated with data that oVirt writes to the attached domain.

The result,I believe, is due to mismatch between ovfstorefile and db.

Comment 2 Maor 2014-09-16 08:38:51 UTC
The exception occurred while the templates has been deleted from the Storage Domain and duo to the engine restart.
The question what happens after the engine is up again.
We should have one of the two scenarios:
1. The compensation will leave the Templates and the disks in the engine
2. The Templates will be removed and their data will be as unregistered entities.

If one of the two didn't happened then we should investigate what was wrong.

Comment 3 Maor 2014-09-16 10:32:18 UTC
Rephrasing the summary after talking with Ori.
The problem is as follow:
1. Add a VM with MSD
2. Detach the Storage Domain from the Data Center
3. Attach the Storage Domain again
4. Add a new disk to the VM on the attached Storage Domain
5. Detach the Storage Domain again
6. The unregistered VM should be overridden by the new VM

This behavior should be similar when Importing this Domain to another setup

Comment 4 Sandro Bonazzola 2014-10-17 12:36:58 UTC
oVirt 3.5 has been released and should include the fix for this issue.