Created attachment 610082 [details]
engine log

Nested exception when trying simultaneously stop several VMs when they already shutting down.
-----------
How to reproduce:
Create several VMs (6-10)
Start them
When their status turns to UP try to stop all of them simultaneously.
Press stop several times.

Reproduces 100%.
Full engine log attached

2012-09-05 17:33:21,681 ERROR [org.ovirt.engine.core.bll.StopVmCommand] (pool-3-thread-47) [22e3d440] Command org.ovirt.engine.core.bll.StopVmCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to DestroyVDS, error = Virtual machine does not exist
2012-09-05 17:33:21,743 ERROR [org.ovirt.engine.core.bll.ShutdownVmCommand] (pool-3-thread-47) [22e3d440] Command org.ovirt.engine.core.bll.ShutdownVmCommand throw exception: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000460: Error checking for a transaction
        at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc-3.1.1.RELEASE.jar:3.1.1.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc-3.1.1.RELEASE.jar:3.1.1.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc-3.1.1.RELEASE.jar:3.1.1.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc-3.1.1.RELEASE.jar:3.1.1.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc-3.1.1.RELEASE.jar:3.1.1.RELEASE]
        at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:155) [engine-dal.jar:]
        at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:121) [engine-dal.jar:]
        at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc-3.1.1.RELEASE.jar:3.1.1.RELEASE]
        at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:124) [engine-dal.jar:]
        at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:75) [engine-dal.jar:]
        at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:66) [engine-dal.jar:]
        at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:58) [engine-dal.jar:]
        at org.ovirt.engine.core.dao.StoragePoolDAODbFacadeImpl.getForVdsGroup(StoragePoolDAODbFacadeImpl.java:159) [engine-dal.jar:]
        at org.ovirt.engine.core.bll.StopVmCommandBase.rollbackQuota(StopVmCommandBase.java:184) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.StopVmCommandBase.setSucceeded(StopVmCommandBase.java:218) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.ShutdownVmCommand.Perform(ShutdownVmCommand.java:83) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.VmOperationCommandBase.ExecuteVmCommand(VmOperationCommandBase.java:34) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.StopVmCommandBase.ExecuteVmCommand(StopVmCommandBase.java:88) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.VmCommand.executeCommand(VmCommand.java:83) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.ExecuteWithoutTransaction(CommandBase.java:819) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:910) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1269) [engine-bll.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:204) [engine-utils.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInRequired(TransactionSupport.java:142) [engine-utils.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:109) [engine-utils.jar:]
        at org.ovirt.engine.core.bll.CommandBase.Execute(CommandBase.java:925) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.ExecuteAction(CommandBase.java:275) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.MultipleActionsRunner.executeValidatedCommands(MultipleActionsRunner.java:182) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.MultipleActionsRunner.RunCommands(MultipleActionsRunner.java:162) [engine-bll.jar:]
        at org.ovirt.engine.core.bll.MultipleActionsRunner$1.run(MultipleActionsRunner.java:84) [engine-bll.jar:]
        at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:64) [engine-utils.jar:]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_03-icedtea]
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) [rt.jar:1.7.0_03-icedtea]
        at java.util.concurrent.FutureTask.run(FutureTask.java:166) [rt.jar:1.7.0_03-icedtea]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [rt.jar:1.7.0_03-icedtea]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [rt.jar:1.7.0_03-icedtea]
        at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_03-icedtea]
:
2012-09-05 17:57:34,312 ERROR [org.ovirt.engine.core.bll.StopVmCommand] (pool-3-thread-40) [68fabe9e] Transaction rolled-back for command: org.ovirt.engine.core.bll.StopVmCommand.
2012-09-05 17:57:34,329 ERROR [org.ovirt.engine.core.bll.ShutdownVmCommand] (pool-3-thread-40) [68fabe9e] Transaction rolled-back for command: org.ovirt.engine.core.bll.ShutdownVmCommand.