Bug 1252958

Summary: [Cinder] Creating a Template from VM + disks on ceph fails "get JDBC Connection"
Product: [oVirt] ovirt-engine Reporter: Ori Gofen <ogofen>
Component: GeneralAssignee: Daniel Erez <derez>
Status: CLOSED CURRENTRELEASE QA Contact: Ori Gofen <ogofen>
Severity: high Docs Contact:
Priority: unspecified    
Version: 3.6.0CC: acanan, amureini, bugs, derez, ecohen, gklein, lsurette, ogofen, rbalakri, yeylon
Target Milestone: ovirt-3.6.0-rcKeywords: Regression
Target Release: 3.6.0Flags: rule-engine: ovirt-3.6.0+
rule-engine: blocker+
ylavi: planning_ack+
rule-engine: devel_ack+
rule-engine: testing_ack+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: storage
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-11-04 13:39:34 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1264677    
Bug Blocks: 1135132    
Attachments:
Description Flags
engine.log
none
log none

Description Ori Gofen 2015-08-12 15:23:52 UTC
Created attachment 1062054 [details]
engine.log

Description of problem:
Tried to create a template from a VM with ceph storage disks, the operation failed.

engine logs:
2015-08-12 17:50:48,898 ERROR [org.ovirt.engine.core.bll.AddVmTemplateCommand] (default task-34) [] Exception: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000460: Error checking for a transaction
        at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE]
        at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:]
        at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:]
        at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE]
        at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:]
        at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:109) [dal.jar:]
        at org.ovirt.engine.core.dao.network.VmNicDaoImpl.getAllForVm(VmNicDaoImpl.java:28) [dal.jar:]
        at org.ovirt.engine.core.bll.AddVmTemplateCommand.addVmInterfaces(AddVmTemplateCommand.java:756) [bll.jar:]
        at org.ovirt.engine.core.bll.AddVmTemplateCommand$2.runInTransaction(AddVmTemplateCommand.java:314) [bll.jar:]
        at org.ovirt.engine.core.bll.AddVmTemplateCommand$2.runInTransaction(AddVmTemplateCommand.java:305) [bll.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:210) [utils.jar:]
        at org.ovirt.engine.core.bll.AddVmTemplateCommand.executeCommand(AddVmTemplateCommand.java:305) [bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1211) [bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1355) [bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1979) [bll.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:]
        at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:]
        at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1392) [bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:374) [bll.jar:]
        at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:475) [bll.jar:]
        at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:457) [bll.jar:]
        at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:413) [bll.jar:]
        at sun.reflect.GeneratedMethodAccessor1046.invoke(Unknown Source) [:1.8.0_45]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_45]
        at java.lang.reflect.Method.invoke(Method.java:497) [rt.jar:1.8.0_45]
        at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)
        at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53)
        at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)
        at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:407)
        at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:70) [wildfly-weld-8.2.0.Final.jar:8.2.0.Final]
        at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:80) [wildfly-weld-8.2.0.Final.jar:8.2.0.Final]
        at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:93) [wildfly-weld-8.2.0.Final.jar:8.2.0.Final]
        at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63)
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)
        at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:407)

This flow was working good thus, flagging it as a regression. 

Version-Release number of selected component (if applicable):
ovirt-engine-3.6.0-0.0.master.20150726172446.git65db93d.el6.noarch

How reproducible:
100%

Steps to Reproduce:
1.create a template from vm+ceph disk

Comment 1 Allon Mureinik 2015-08-13 09:07:45 UTC
Seems like the database just failed mid-operation, which is completely unrelated to the cinder integration (see, e.g., similar errors on jobs and audit logs).

Daniel - please confirm and move to infra or refute and fix.

Comment 2 Daniel Erez 2015-08-17 08:43:07 UTC
Couldn't reproduce on latest build - flow seems to work fine.
@Ori - have you tried it on dev env or latest 3.6 build? Can you please check that the DB has been upgraded properly?
Thanks.

Comment 3 Ori Gofen 2015-08-30 13:09:51 UTC
Created attachment 1068426 [details]
log

This is reproduced on rhevm-3.6.0-0.12.master.el6.noarch

Comment 4 Daniel Erez 2015-09-01 06:31:30 UTC
After investigating the issue with Ori, it seems that the flow works correctly on latest stable oVirt build. @Ori - could you please report whether the issue is reproduced once rhevm-3.6.0-0.13 is out? Thanks!

Comment 5 Ori Gofen 2015-09-02 13:18:21 UTC
I believe the best way to monitor this bug is to move it to ON_QA, since we already know the fix has been merged to current stable master thus insuring this one will pop up the minute 13 build comes out

Comment 6 Red Hat Bugzilla Rules Engine 2015-09-22 07:43:53 UTC
This bug report has Keywords: Regression or TestBlocker.
Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.

Comment 7 Ori Gofen 2015-10-19 12:26:45 UTC
verified on rhevm-3.6.0.1-0.1.el6.noarch

Comment 8 Sandro Bonazzola 2015-11-04 13:39:34 UTC
oVirt 3.6.0 has been released on November 4th, 2015 and should fix this issue.
If problems still persist, please open a new BZ and reference this one.