Bug 1475063
Summary: | hosted self engine can't start | ||
---|---|---|---|
Product: | [oVirt] ovirt-engine | Reporter: | hsiuming <hsiuming> |
Component: | General | Assignee: | bugs <bugs> |
Status: | CLOSED INSUFFICIENT_DATA | QA Contact: | |
Severity: | high | Docs Contact: | |
Priority: | unspecified | ||
Version: | 4.1.1 | CC: | bugs, hsiuming, sabose |
Target Milestone: | --- | ||
Target Release: | --- | ||
Hardware: | x86_64 | ||
OS: | Linux | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | If docs needed, set a value | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2017-12-04 06:55:04 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | Gluster | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
hsiuming
2017-07-26 00:48:19 UTC
And the engine log in the engine VM shows : 2017-07-25 22:36:28,030-04 ERROR [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 54) [] Failed to initialize backend: org.jboss.weld.exceptions.WeldException: WELD-000049: Unable to invoke protected void org.ovirt.engine.core.bll.scheduling.AffinityRulesEnforcementManager.wakeup() on org.ovirt.engine.core.bll.scheduling.AffinityRulesEnforcementManager@5ea69caf at org.jboss.weld.injection.producer.DefaultLifecycleCallbackInvoker.invokeMethods(DefaultLifecycleCallbackInvoker.java:100) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] at org.jboss.weld.injection.producer.DefaultLifecycleCallbackInvoker.postConstruct(DefaultLifecycleCallbackInvoker.java:81) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] at org.jboss.weld.injection.producer.BasicInjectionTarget.postConstruct(BasicInjectionTarget.java:126) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] at org.jboss.weld.bean.ManagedBean.create(ManagedBean.java:162) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] at org.jboss.weld.context.AbstractContext.get(AbstractContext.java:96) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] at org.jboss.weld.bean.ContextualInstanceStrategy$DefaultContextualInstanceStrategy.get(ContextualInstanceStrategy.java:101) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] at org.jboss.weld.bean.ContextualInstanceStrategy$ApplicationScopedContextualInstanceStrategy.get(ContextualInstanceStrategy.java:141) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] at org.jboss.weld.bean.ContextualInstance.get(ContextualInstance.java:50) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] at org.jboss.weld.manager.BeanManagerImpl.getReference(BeanManagerImpl.java:742) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] at org.jboss.weld.bean.builtin.InstanceImpl.getBeanInstance(InstanceImpl.java:189) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] at org.jboss.weld.bean.builtin.InstanceImpl.get(InstanceImpl.java:100) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] at org.ovirt.engine.core.bll.ServiceLoader.load(ServiceLoader.java:34) [bll.jar:] at org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean.create(InitBackendServicesOnStartupBean.java:96) [bll.jar:] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_121] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_121] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_121] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_121] at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:96) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doLifecycleInterception(Jsr299BindingsInterceptor.java:114) [wildfly-weld-10.1.0.Final.jar:10.1.0.Final] at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:103) [wildfly-weld-10.1.0.Final.jar:10.1.0.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437) at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:83) [wildfly-weld-10.1.0.Final.jar:10.1.0.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:53) [wildfly-weld-10.1.0.Final.jar:10.1.0.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.AroundConstructInterceptorFactory$1.processInvocation(AroundConstructInterceptorFactory.java:28) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.weld.injection.WeldInterceptorInjectionInterceptor.processInvocation(WeldInterceptorInjectionInterceptor.java:56) [wildfly-weld-10.1.0.Final.jar:10.1.0.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.weld.ejb.Jsr299BindingsCreateInterceptor.processInvocation(Jsr299BindingsCreateInterceptor.java:100) [wildfly-weld-10.1.0.Final.jar:10.1.0.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:275) [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final] at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:349) [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final] at org.jboss.as.ejb3.tx.LifecycleCMTTxInterceptor.processInvocation(LifecycleCMTTxInterceptor.java:68) [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.weld.injection.WeldInjectionContextInterceptor.processInvocation(WeldInjectionContextInterceptor.java:43) [wildfly-weld-10.1.0.Final.jar:10.1.0.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45) [wildfly-ee-10.1.0.Final.jar:10.1.0.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.singleton.StartupCountDownInterceptor.processInvocation(StartupCountDownInterceptor.java:25) [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:161) at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:134) at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:88) at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:124) [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final] at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:138) [wildfly-ejb3-10.1.0.Final.jar:10.1.0.Final] at org.jboss.as.ee.component.ComponentStartService$1.run(ComponentStartService.java:54) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_121] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_121] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_121] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_121] at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_121] at org.jboss.threads.JBossThread.run(JBossThread.java:320) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_121] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_121] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_121] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_121] at org.jboss.weld.injection.producer.DefaultLifecycleCallbackInvoker.invokeMethods(DefaultLifecycleCallbackInvoker.java:98) [weld-core-impl-2.3.5.Final.jar:2.3.5.Final] ... 65 more Caused by: org.springframework.jdbc.UncategorizedSQLException: CallableStatementCallback; uncategorized SQLException for SQL [{call insertauditlog(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}]; SQL state [XX001]; error code [0]; ERROR: could not read block 20 in file "base/16385/19110": read only 0 of 8192 bytes Where: SQL statement "INSERT INTO audit_log ( LOG_TIME, log_type, log_type_name, severity, message, user_id, user_name, vds_id, vds_name, vm_id, vm_name, vm_template_id, vm_template_name, storage_pool_id, storage_pool_name, storage_domain_id, storage_domain_name, cluster_id, cluster_name, correlation_id, job_id, quota_id, quota_name, gluster_volume_id, gluster_volume_name, call_stack, brick_id, brick_path ) VALUES ( v_log_time, v_log_type, v_log_type_name, v_severity, v_message, v_user_id, v_user_name, v_vds_id, v_vds_name, v_vm_id, v_vm_name, v_vm_template_id, v_vm_template_name, v_storage_pool_id, v_storage_pool_name, v_storage_domain_id, v_storage_domain_name, v_cluster_id, v_cluster_name, v_correlation_id, v_job_id, v_quota_id, v_quota_name, v_gluster_volume_id, v_gluster_volume_name, v_call_stack, v_brick_id, v_brick_path )" PL/pgSQL function insertauditlog(integer,timestamp with time zone,integer,character varying,integer,text,uuid,character varying,uuid,character varying,uuid,character varying,uuid,character varying,uuid,character varying,uuid,character varying,uuid,character varying,uuid,character varying,character varying,uuid,uuid,character varying,text,boolean,uuid,text) line 9 at SQL statement; nested exception is org.postgresql.util.PSQLException: ERROR: could not read block 20 in file "base/16385/19110": read only 0 of 8192 bytes Where: SQL statement "INSERT INTO audit_log ( LOG_TIME, log_type, log_type_name, severity, message, user_id, user_name, vds_id, vds_name, vm_id, vm_name, vm_template_id, vm_template_name, storage_pool_id, storage_pool_name, storage_domain_id, storage_domain_name, cluster_id, cluster_name, correlation_id, job_id, quota_id, quota_name, gluster_volume_id, gluster_volume_name, call_stack, brick_id, brick_path ) VALUES ( v_log_time, v_log_type, v_log_type_name, v_severity, v_message, v_user_id, v_user_name, v_vds_id, v_vds_name, v_vm_id, v_vm_name, v_vm_template_id, v_vm_template_name, v_storage_pool_id, v_storage_pool_name, v_storage_domain_id, v_storage_domain_name, v_cluster_id, v_cluster_name, v_correlation_id, v_job_id, v_quota_id, v_quota_name, v_gluster_volume_id, v_gluster_volume_name, v_call_stack, v_brick_id, v_brick_path )" PL/pgSQL function insertauditlog(integer,timestamp with time zone,integer,character varying,integer,text,uuid,character varying,uuid,character varying,uuid,character varying,uuid,character varying,uuid,character varying,uuid,character varying,uuid,character varying,uuid,character varying,character varying,uuid,uuid,character varying,text,boolean,uuid,text) line 9 at SQL statement at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:84) [spring-jdbc.jar:4.2.4.RELEASE] at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81) [spring-jdbc.jar:4.2.4.RELEASE] at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81) [spring-jdbc.jar:4.2.4.RELEASE] at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1094) [spring-jdbc.jar:4.2.4.RELEASE] at org.springframework.jdbc.core.JdbcTemplate.call(JdbcTemplate.java:1130) [spring-jdbc.jar:4.2.4.RELEASE] at org.springframework.jdbc.core.simple.AbstractJdbcCall.executeCallInternal(AbstractJdbcCall.java:405) [spring-jdbc.jar:4.2.4.RELEASE] at org.springframework.jdbc.core.simple.AbstractJdbcCall.doExecute(AbstractJdbcCall.java:365) [spring-jdbc.jar:4.2.4.RELEASE] at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:198) [spring-jdbc.jar:4.2.4.RELEASE] at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:135) [dal.jar:] at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:130) [dal.jar:] at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeModification(SimpleJdbcCallsHandler.java:76) [dal.jar:] at org.ovirt.engine.core.dao.AuditLogDaoImpl.save(AuditLogDaoImpl.java:169) [dal.jar:] at org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector.saveToDb(AuditLogDirector.java:95) [dal.jar:] at org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector.log(AuditLogDirector.java:81) [dal.jar:] at org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector.log(AuditLogDirector.java:70) [dal.jar:] at org.ovirt.engine.core.bll.scheduling.AffinityRulesEnforcementManager.wakeup(AffinityRulesEnforcementManager.java:51) [bll.jar:] ... 70 more Caused by: org.postgresql.util.PSQLException: ERROR: could not read block 20 in file "base/16385/19110": read only 0 of 8192 bytes Where: SQL statement "INSERT INTO audit_log ( LOG_TIME, log_type, log_type_name, severity, message, user_id, user_name, vds_id, vds_name, vm_id, vm_name, vm_template_id, vm_template_name, storage_pool_id, storage_pool_name, storage_domain_id, storage_domain_name, cluster_id, cluster_name, correlation_id, job_id, quota_id, quota_name, gluster_volume_id, gluster_volume_name, call_stack, brick_id, brick_path ) VALUES ( v_log_time, v_log_type, v_log_type_name, v_severity, v_message, v_user_id, v_user_name, v_vds_id, v_vds_name, v_vm_id, v_vm_name, v_vm_template_id, v_vm_template_name, v_storage_pool_id, v_storage_pool_name, v_storage_domain_id, v_storage_domain_name, v_cluster_id, v_cluster_name, v_correlation_id, v_job_id, v_quota_id, v_quota_name, v_gluster_volume_id, v_gluster_volume_name, v_call_stack, v_brick_id, v_brick_path )" PL/pgSQL function insertauditlog(integer,timestamp with time zone,integer,character varying,integer,text,uuid,character varying,uuid,character varying,uuid,character varying,uuid,character varying,uuid,character varying,uuid,character varying,uuid,character varying,uuid,character varying,character varying,uuid,uuid,character varying,text,boolean,uuid,text) line 9 at SQL statement at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2157) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1886) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:555) at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:417) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:410) at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.execute(CachedPreparedStatement.java:303) at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.execute(WrappedPreparedStatement.java:442) at org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1133) [spring-jdbc.jar:4.2.4.RELEASE] at org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1130) [spring-jdbc.jar:4.2.4.RELEASE] at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1078) [spring-jdbc.jar:4.2.4.RELEASE] ... 82 more It seems like sql issue? org.postgresql.util.PSQLException: ERROR: could not read block 20 in file "base/16385/19110": read only 0 of 8192 bytes Suggests an issue with database or filesystem. Could you provide the gluster mount logs for engine volume as well as gluster brick logs? Are you able to provide requested info? Closing this as there's no data provided to look into this further The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days |