Bug 1249671

Summary: Can't start a VM after its creation on FC22
Product: [oVirt] ovirt-engine Reporter: Sandro Bonazzola <sbonazzo>
Component: GeneralAssignee: Alona Kaplan <alkaplan>
Status: CLOSED CURRENTRELEASE QA Contact: Pavel Stehlik <pstehlik>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: ---CC: alkaplan, bazulay, bugs, dcaroest, ecohen, eedri, emesika, gklein, lsurette, myakove, oourfali, rbalakri, sbonazzo, stirabos, yeylon, ylavi
Target Milestone: ovirt-3.6.0-rcKeywords: Regression
Target Release: 3.6.0Flags: ylavi: ovirt-3.6.0?
rule-engine: blocker?
ylavi: planning_ack+
rule-engine: devel_ack+
ylavi: testing_ack?
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: network
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-11-11 10:26:09 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Network RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
engine logs none

Description Sandro Bonazzola 2015-08-03 14:28:45 UTC
Using ovirt-engine-3.6.0-0.0.master.20150802175852.git8c45eb7.fc22.noarch


Caused by: org.postgresql.util.PSQLException: ERROR: function gethostnetworksbycluster(uuid, unknown, character varying) does not exist
  Hint: No function matches the given name and argument types. You might need to add explicit type casts.
  Position: 16
        at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2198)
        at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1927)
        at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:565)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:420)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:305)
        at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.executeQuery(CachedPreparedStatement.java:111)
        at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:462)
        at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:646) [spring-jdbc.jar:]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:589) [spring-jdbc.jar:]
        ... 162 more


2015-08-03 14:23:24,723 INFO  [org.ovirt.engine.core.bll.RunVmCommand] (default task-2) [ef7263a] Lock freed to object 'EngineLock:{exclusiveLocks='[8d6a72bb-11bc-4465-929a-65a9b7097ad0=<VM, ACTION_TYPE_FAILED_O
BJECT_LOCKED>]', sharedLocks='null'}'
2015-08-03 14:23:33,075 INFO  [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-3) [76fc9803] Lock Acquired to object 'EngineLock:{exclusiveLocks='[8d6a72bb-11bc-4465-929a-65a9b7097ad0=<VM, ACTION_TYPE_
FAILED_OBJECT_LOCKED>]', sharedLocks='null'}'
2015-08-03 14:23:33,143 INFO  [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (default task-3) [76fc9803] START, IsVmDuringInitiatingVDSCommand( IsVmDuringInitiatingVDSCommandParameters:{runAsyn
c='true', vmId='8d6a72bb-11bc-4465-929a-65a9b7097ad0'}), log id: 38a362d9
2015-08-03 14:23:33,144 INFO  [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (default task-3) [76fc9803] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 38a362d9
2015-08-03 14:23:33,174 ERROR [org.ovirt.engine.core.bll.RunVmOnceCommand] (default task-3) [76fc9803] Data access error during CanDoActionFailure.: org.springframework.jdbc.BadSqlGrammarException: PreparedState
mentCallback; bad SQL grammar [select * from  gethostnetworksbycluster(?, ?, ?)]; nested exception is org.postgresql.util.PSQLException: ERROR: function gethostnetworksbycluster(uuid, unknown, character varying) does not exist
  Hint: No function matches the given name and argument types. You might need to add explicit type casts.
  Position: 16
        at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:97) [spring-jdbc.jar:]
        at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:]
        at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:]
        at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:80) [spring-jdbc.jar:]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:605) [spring-jdbc.jar:]
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:639) [spring-jdbc.jar:]
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:668) [spring-jdbc.jar:]
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:708) [spring-jdbc.jar:]

Comment 1 Oved Ourfali 2015-08-03 15:58:52 UTC
This is a network issue. Not infra. 
Setting whiteboard accordingly.

Comment 2 Oved Ourfali 2015-08-03 16:21:54 UTC
Sandro, we're there any issues during the setup? As I guess this function is added on regular setup database scripts, unless it was renamed or deleted.

Comment 3 Alona Kaplan 2015-08-03 16:33:02 UTC
Sandro, can you please attach the full log?

Couldn't reproduce the issue since I couldn't even add a vm via the gui-

2015-08-03 19:16:42,436 ERROR [org.ovirt.engine.core.bll.AddVmCommand] (default task-70) [] Command 'org.ovirt.engine.core.bll.AddVmCommand' failed: CallableStatementCallback; bad SQL grammar [{call insertvmstatic(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}]; nested exception is org.postgresql.util.PSQLException: ERROR: column "dedicated_vm_for_vds" is of type uuid but expression is of type text
  Hint: You will need to rewrite or cast the expression.
  Where: PL/pgSQL function insertvmstatic(character varying,text,integer,integer,integer,uuid,uuid,character varying,uuid,timestamp with time zone,integer,boolean,boolean,boolean,integer,integer,integer,character varying,boolean,boolean,boolean,boolean,character varying,text,boolean,integer,integer,integer,integer,integer,integer,character varying,integer,character varying,character varying,character varying,integer,character varying,character varying,integer,uuid,character varying,boolean,boolean,character varying,boolean,uuid,uuid,uuid,uuid,character varying,integer,integer,smallint,character varying,boolean,character varying,boolean,boolean,uuid,boolean,boolean,character varying,character varying,uuid,uuid,character varying) line 8 at SQL statement
2015-08-03 19:16:42,436 ERROR [org.ovirt.engine.core.bll.AddVmCommand] (default task-70) [] Exception: org.springframework.jdbc.BadSqlGrammarException: CallableStatementCallback; bad SQL grammar [{call insertvmstatic(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}]; nested exception is org.postgresql.util.PSQLException: ERROR: column "dedicated_vm_for_vds" is of type uuid but expression is of type text
  Hint: You will need to rewrite or cast the expression.
  Where: PL/pgSQL function insertvmstatic(character varying,text,integer,integer,integer,uuid,uuid,character varying,uuid,timestamp with time zone,integer,boolean,boolean,boolean,integer,integer,integer,character varying,boolean,boolean,boolean,boolean,character varying,text,boolean,integer,integer,integer,integer,integer,integer,character varying,integer,character varying,character varying,character varying,integer,character varying,character varying,integer,uuid,character varying,boolean,boolean,character varying,boolean,uuid,uuid,uuid,uuid,character varying,integer,integer,smallint,character varying,boolean,character varying,boolean,boolean,uuid,boolean,boolean,character varying,character varying,uuid,uuid,character varying) line 8 at SQL statement
	at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:233) [spring-jdbc.jar:3.1.1.RELEASE]
	at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE]
	at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1030) [spring-jdbc.jar:3.1.1.RELEASE]
	at org.springframework.jdbc.core.JdbcTemplate.call(JdbcTemplate.java:1064) [spring-jdbc.jar:3.1.1.RELEASE]
	at org.springframework.jdbc.core.simple.AbstractJdbcCall.executeCallInternal(AbstractJdbcCall.java:388) [spring-jdbc.jar:3.1.1.RELEASE]
	at org.springframework.jdbc.core.simple.AbstractJdbcCall.doExecute(AbstractJdbcCall.java:351) [spring-jdbc.jar:3.1.1.RELEASE]
	at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE]
	at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:]
	at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:142) [dal.jar:]
	at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeModification(SimpleJdbcCallsHandler.java:80) [dal.jar:]
	at org.ovirt.engine.core.dao.DefaultGenericDaoDbFacade.save(DefaultGenericDaoDbFacade.java:93) [dal.jar:]
	at org.ovirt.engine.core.bll.AddVmCommand.addVmStatic(AddVmCommand.java:1097) [bll.jar:]
	at org.ovirt.engine.core.bll.AddVmCommand$1.runInTransaction(AddVmCommand.java:882) [bll.jar:]
	at org.ovirt.engine.core.bll.AddVmCommand$1.runInTransaction(AddVmCommand.java:878) [bll.jar:]
	at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:210) [utils.jar:]
	at org.ovirt.engine.core.bll.AddVmCommand.executeVmCommand(AddVmCommand.java:878) [bll.jar:]
	at org.ovirt.engine.core.bll.VmCommand.executeCommand(VmCommand.java:104) [bll.jar:]
	at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1211) [bll.jar:]
	at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1355) [bll.jar:]
	at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1979) [bll.jar:]
	at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:]
	at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:]
	at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1392) [bll.jar:]
	at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:374) [bll.jar:]
	at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:475) [bll.jar:]
	at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:457) [bll.jar:]
	at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:413) [bll.jar:]
	at sun.reflect.GeneratedMethodAccessor711.invoke(Unknown Source) [:1.7.0_79]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_79]
	at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_79]
	at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52)
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)
	at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53)
	at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63)
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)
	at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:407)
	at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:70) [wildfly-weld-8.2.0.Final.jar:8.2.0.Final]
	at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:80) [wildfly-weld-8.2.0.Final.jar:8.2.0.Final]
	at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:93) [wildfly-weld-8.2.0.Final.jar:8.2.0.Final]
	at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63)
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)
	at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:407)
	at org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerInterceptor.aroundInvoke(CorrelationIdTrackerInterceptor.java:13) [bll.jar:]
	at sun.reflect.GeneratedMethodAccessor81.invoke(Unknown Source) [:1.7.0_79]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_79]
	at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_79]
	at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89)
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)
	at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53)
	at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63)
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)
	at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [wildfly-ejb3-8.2.0.Final.jar:8.2.0.Final]
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)
	at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:407)
	at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:55) [weld-core-impl-2.2.6.Final.jar:2014-10-03 10:05]
	at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:83) [wildfly-weld-8.2.0.Final.jar:8.2.0.Final]
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)
	at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45) [wildfly-ee-8.2.0.Final.jar:8.2.0.Final]
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)
	at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21)
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)
	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
	at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53)
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)
	at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [wildfly-ejb3-8.2.0.Final.jar:8.2.0.F

Comment 4 Sandro Bonazzola 2015-08-04 06:45:57 UTC
(In reply to Alona Kaplan from comment #3)
> Sandro, can you please attach the full log?
> 
> Couldn't reproduce the issue since I couldn't even add a vm via the gui-

Please open a separate BZ for this issue. Will attach logs as soon I'll get access again to the system.

Comment 5 Sandro Bonazzola 2015-08-04 07:27:54 UTC
Simone, please attach logs from the host we used yesterday

Comment 6 Simone Tiraboschi 2015-08-04 07:33:58 UTC
Alona is working on that host.

Comment 7 Alona Kaplan 2015-08-04 09:35:58 UTC
(In reply to Sandro Bonazzola from comment #4)
> (In reply to Alona Kaplan from comment #3)
> > Sandro, can you please attach the full log?
> > 
> > Couldn't reproduce the issue since I couldn't even add a vm via the gui-
> 
> Please open a separate BZ for this issue. Will attach logs as soon I'll get
> access again to the system.

Ignore it, is was a problem with my environment.

Comment 8 Simone Tiraboschi 2015-08-04 09:51:28 UTC
Created attachment 1058997 [details]
engine logs

Comment 9 Red Hat Bugzilla Rules Engine 2015-09-22 07:43:47 UTC
This bug report has Keywords: Regression or TestBlocker.
Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.

Comment 10 Meni Yakove 2015-11-11 10:19:14 UTC
can you verify this bug?