Bug 854694

Summary: Nested exception when trying simultaneously stop several VMs when they already shutting down.
Product: Red Hat Enterprise Virtualization Manager Reporter: Leonid Natapov <lnatapov>
Component: ovirt-engineAssignee: Omer Frenkel <ofrenkel>
Status: CLOSED DUPLICATE QA Contact: Pavel Stehlik <pstehlik>
Severity: high Docs Contact:
Priority: unspecified    
Version: 3.1.0CC: acathrow, hateya, iheim, jkt, lpeer, michal.skrivanek, Rhev-m-bugs, yeylon
Target Milestone: ---   
Target Release: 3.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: virt
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-12-09 07:28:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
engine log none

Description Leonid Natapov 2012-09-05 15:02:03 UTC
Created attachment 610082 [details]
engine log

Nested exception when trying simultaneously stop several VMs when they already shutting down.
-----------
How to reproduce:
Create several VMs (6-10)
Start them
When their status turns to UP try to stop all of them simultaneously.
Press stop several times.

Reproduces 100%.
Full engine log attached

2012-09-05 17:33:21,681 ERROR [org.ovirt.engine.core.bll.StopVmCommand] (pool-3-thread-47) [22e3d440] Command org.ovirt.engine.core.bll.StopVmCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to DestroyVDS, error = Virtual machine does not exist
2012-09-05 17:33:21,743 ERROR [org.ovirt.engine.core.bll.ShutdownVmCommand] (pool-3-thread-47) [22e3d440] Command org.ovirt.engine.core.bll.ShutdownVmCommand throw exception: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000460: Error checking for a transaction
        at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc-3.1.1.RELEASE.jar:3.1.1.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc-3.1.1.RELEASE.jar:3.1.1.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc-3.1.1.RELEASE.jar:3.1.1.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc-3.1.1.RELEASE.jar:3.1.1.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc-3.1.1.RELEASE.jar:3.1.1.RELEASE]
        at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:155) [engine-dal.jar:]
        at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:121) [engine-dal.jar:]
        at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc-3.1.1.RELEASE.jar:3.1.1.RELEASE]
        at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:124) [engine-dal.jar:]
        at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:75) [engine-dal.jar:]
        at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:66) [engine-dal.jar:]
        at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:58) [engine-dal.jar:]
        at org.ovirt.engine.core.dao.StoragePoolDAODbFacadeImpl.getForVdsGroup(StoragePoolDAODbFacadeImpl.java:159) [engine-dal.jar:]
        at org.ovirt.engine.core.bll.StopVmCommandBase.rollbackQuota(StopVmCommandBase.java:184) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.StopVmCommandBase.setSucceeded(StopVmCommandBase.java:218) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.ShutdownVmCommand.Perform(ShutdownVmCommand.java:83) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.VmOperationCommandBase.ExecuteVmCommand(VmOperationCommandBase.java:34) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.StopVmCommandBase.ExecuteVmCommand(StopVmCommandBase.java:88) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.VmCommand.executeCommand(VmCommand.java:83) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.ExecuteWithoutTransaction(CommandBase.java:819) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:910) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1269) [engine-bll.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:204) [engine-utils.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInRequired(TransactionSupport.java:142) [engine-utils.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:109) [engine-utils.jar:]
        at org.ovirt.engine.core.bll.CommandBase.Execute(CommandBase.java:925) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.ExecuteAction(CommandBase.java:275) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.MultipleActionsRunner.executeValidatedCommands(MultipleActionsRunner.java:182) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.MultipleActionsRunner.RunCommands(MultipleActionsRunner.java:162) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.MultipleActionsRunner$1.run(MultipleActionsRunner.java:84) [engine-bll.jar:]
        at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:64) [engine-utils.jar:]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_03-icedtea]
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_03-icedtea]
        at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_03-icedtea]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [rt.jar:1.7.0_03-icedtea]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [rt.jar:1.7.0_03-icedtea]
        at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_03-icedtea]
:
2012-09-05 17:57:34,312 ERROR [org.ovirt.engine.core.bll.StopVmCommand] (pool-3-thread-40) [68fabe9e] Transaction rolled-back for command: org.ovirt.engine.core.bll.StopVmCommand.
2012-09-05 17:57:34,329 ERROR [org.ovirt.engine.core.bll.ShutdownVmCommand] (pool-3-thread-40) [68fabe9e] Transaction rolled-back for command: org.ovirt.engine.core.bll.ShutdownVmCommand.

Comment 1 Michal Skrivanek 2012-09-19 13:57:15 UTC
error is harmless. we should probably add a check to not send shutdown again until the previous one finished. There should be some timeout as well, I guess.

Comment 2 Michal Skrivanek 2013-07-09 12:31:24 UTC
per scrub mtg push to 3.4

Comment 3 Itamar Heim 2013-12-08 07:43:53 UTC
still planned for 3.4?

Comment 4 Michal Skrivanek 2013-12-09 07:28:33 UTC
should be addressed by http://gerrit.ovirt.org/#/c/21937/

*** This bug has been marked as a duplicate of bug 1027297 ***