Bug 1580319
| Summary: | [3.6] Engine is unreachable via UI/REST-API | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Virtualization Manager | Reporter: | Elad <ebenahar> | ||||
| Component: | ovirt-engine | Assignee: | Ravi Nori <rnori> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Petr Matyáš <pmatyas> | ||||
| Severity: | urgent | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 3.6.10 | CC: | amarchuk, amashah, bugs, ebenahar, kmashalk, lsurette, lsvaty, mkalinin, mperina, mtessun, rbalakri, Rhev-m-bugs, srevivo, ykaul | ||||
| Target Milestone: | ovirt-3.6.13 | Keywords: | Regression, ZStream | ||||
| Target Release: | --- | ||||||
| Hardware: | x86_64 | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | |||||||
| : | 1580398 1623544 (view as bug list) | Environment: | |||||
| Last Closed: | 2018-05-24 18:26:35 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | Infra | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | 1580398 | ||||||
| Bug Blocks: | 1623544 | ||||||
| Attachments: |
|
||||||
Below error started the issue:
2018-05-19 23:26:31,255 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 51) MSC000001: Failed to start service jboss.deployment.subunit."engine.ear"."bll.jar".component.InitBackendServicesOnStartupBean.START: org.jboss.msc.service.StartException in service jboss.deployment.subunit."engine.ear"."bll.jar".component.InitBackendServicesOnStartupBean.START: java.lang.IllegalStateException: JBAS011048: Failed to construct component instance
at org.jboss.as.ee.component.ComponentStartService$1.run(ComponentStartService.java:57) [jboss-as-ee.jar:7.5.20.Final-redhat-1]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_121]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_121]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_121]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_121]
at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_121]
at org.jboss.threads.JBossThread.run(JBossThread.java:122)
Caused by: java.lang.IllegalStateException: JBAS011048: Failed to construct component instance
at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:164) [jboss-as-ee.jar:7.5.20.Final-redhat-1]
at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:135) [jboss-as-ee.jar:7.5.20.Final-redhat-1]
at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:90) [jboss-as-ee.jar:7.5.20.Final-redhat-1]
at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:122)
at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:137)
at org.jboss.as.ee.component.ComponentStartService$1.run(ComponentStartService.java:54) [jboss-as-ee.jar:7.5.20.Final-redhat-1]
... 6 more
Caused by: javax.ejb.EJBException: org.apache.commons.lang.SerializationException: org.codehaus.jackson.map.JsonMappingException: Illegal type (org.ovirt.engine.core.common.action.AddImageFromScratchParameters) to deserialize: prevented for security reasons
at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleExceptionInOurTx(CMTTxInterceptor.java:191)
at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:281)
at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:353)
at org.jboss.as.ejb3.tx.LifecycleCMTTxInterceptor.processInvocation(LifecycleCMTTxInterceptor.java:67)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]
at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]
at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:70) [jboss-invocation.jar:1.1.3.Final-redhat-1]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]
at org.jboss.as.ejb3.component.singleton.StartupCountDownInterceptor.processInvocation(StartupCountDownInterceptor.java:25)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]
at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.3.Final-redhat-1]
at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:162) [jboss-as-ee.jar:7.5.20.Final-redhat-1]
... 11 more
Caused by: org.apache.commons.lang.SerializationException: org.codehaus.jackson.map.JsonMappingException: Illegal type (org.ovirt.engine.core.common.action.AddImageFromScratchParameters) to deserialize: prevented for security reasons
at org.ovirt.engine.core.utils.serialization.json.JsonObjectDeserializer.readJsonString(JsonObjectDeserializer.java:101)
at org.ovirt.engine.core.utils.serialization.json.JsonObjectDeserializer.deserialize(JsonObjectDeserializer.java:57)
at org.ovirt.engine.core.dao.CommandEntityDaoImpl.deserializeParameters(CommandEntityDaoImpl.java:150)
at org.ovirt.engine.core.dao.CommandEntityDaoImpl.access$000(CommandEntityDaoImpl.java:32)
at org.ovirt.engine.core.dao.CommandEntityDaoImpl$1.mapRow(CommandEntityDaoImpl.java:71)
at org.ovirt.engine.core.dao.CommandEntityDaoImpl$1.mapRow(CommandEntityDaoImpl.java:58)
at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:92)
at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:1)
at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:649)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706)
at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154)
at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120)
at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181)
at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147)
at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:109)
at org.ovirt.engine.core.dao.DefaultReadDao.getAll(DefaultReadDao.java:77)
at org.ovirt.engine.core.bll.tasks.CommandsCacheImpl.initializeCache(CommandsCacheImpl.java:33)
at org.ovirt.engine.core.bll.tasks.CommandsCacheImpl.get(CommandsCacheImpl.java:50)
at org.ovirt.engine.core.bll.tasks.CommandCoordinatorImpl.getCommandEntity(CommandCoordinatorImpl.java:132)
at org.ovirt.engine.core.bll.tasks.CoCoAsyncTaskHelper.getCommandEntity(CoCoAsyncTaskHelper.java:160)
at org.ovirt.engine.core.bll.tasks.CoCoAsyncTaskHelper.getAllAsyncTasksFromDb(CoCoAsyncTaskHelper.java:153)
at org.ovirt.engine.core.bll.tasks.CommandCoordinatorImpl.getAllAsyncTasksFromDb(CommandCoordinatorImpl.java:292)
at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.initAsyncTaskManager(AsyncTaskManager.java:113)
at org.ovirt.engine.core.bll.tasks.CommandCoordinatorUtil.initAsyncTaskManager(CommandCoordinatorUtil.java:49)
at org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean.create(InitBackendServicesOnStartupBean.java:60)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_121]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_121]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_121]
at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_121]
at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:96) [jboss-as-ee.jar:7.5.20.Final-redhat-1]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]
at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.3.Final-redhat-1]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]
at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doLifecycleInterception(Jsr299BindingsInterceptor.java:116)
at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:107)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]
at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:93)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]
at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:67)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]
at org.jboss.as.ee.component.ComponentInstantiatorInterceptor.processInvocation(ComponentInstantiatorInterceptor.java:76) [jboss-as-ee.jar:7.5.20.Final-redhat-1]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]
at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.3.Final-redhat-1]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]
at org.jboss.as.weld.ejb.Jsr299BindingsCreateInterceptor.processInvocation(Jsr299BindingsCreateInterceptor.java:92)
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]
at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.5.20.Final-redhat-1]
at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]
at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279)
... 22 more
Caused by: org.codehaus.jackson.map.JsonMappingException: Illegal type (org.ovirt.engine.core.common.action.AddImageFromScratchParameters) to deserialize: prevented for security reasons
at org.codehaus.jackson.map.deser.BeanDeserializerFactory.checkLegalTypes(BeanDeserializerFactory.java:1521)
at org.codehaus.jackson.map.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:655)
at org.codehaus.jackson.map.deser.StdDeserializerProvider._createDeserializer(StdDeserializerProvider.java:401)
at org.codehaus.jackson.map.deser.StdDeserializerProvider._createAndCache2(StdDeserializerProvider.java:310)
at org.codehaus.jackson.map.deser.StdDeserializerProvider._createAndCacheValueDeserializer(StdDeserializerProvider.java:290)
at org.codehaus.jackson.map.deser.StdDeserializerProvider.findValueDeserializer(StdDeserializerProvider.java:159)
at org.codehaus.jackson.map.deser.StdDeserializerProvider.findTypedValueDeserializer(StdDeserializerProvider.java:180)
at org.codehaus.jackson.map.ObjectMapper._findRootDeserializer(ObjectMapper.java:2829)
at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2728)
at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1863)
at org.ovirt.engine.core.utils.serialization.json.JsonObjectDeserializer.readJsonString(JsonObjectDeserializer.java:99)
... 73 more
2018-05-19 23:26:31,266 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) JBAS014612: Operation ("deploy") failed - address: ([("deployment" => "engine.ear")]) - failure description: {"JBAS014671: Failed services" => {"jboss.deployment.subunit.\"engine.ear\".\"bll.jar\".component.InitBackendServicesOnStartupBean.START" => "org.jboss.msc.service.StartException in service jboss.deployment.subunit.\"engine.ear\".\"bll.jar\".component.InitBackendServicesOnStartupBean.START: java.lang.IllegalStateException: JBAS011048: Failed to construct component instance
Caused by: java.lang.IllegalStateException: JBAS011048: Failed to construct component instance
Caused by: javax.ejb.EJBException: org.apache.commons.lang.SerializationException: org.codehaus.jackson.map.JsonMappingException: Illegal type (org.ovirt.engine.core.common.action.AddImageFromScratchParameters) to deserialize: prevented for security reasons
Caused by: org.apache.commons.lang.SerializationException: org.codehaus.jackson.map.JsonMappingException: Illegal type (org.ovirt.engine.core.common.action.AddImageFromScratchParameters) to deserialize: prevented for security reasons
Caused by: org.codehaus.jackson.map.JsonMappingException: Illegal type (org.ovirt.engine.core.common.action.AddImageFromScratchParameters) to deserialize: prevented for security reasons"}}
This is similar error as seen in BZ1577407, could you please try with EAP 6.4.19?
I've just verified that downgrading jackson library included in EAP 6.4.19 fixed the issue, I will open EAP bug and attach it to this one and BZ1577407 Verified on rhevm-3.6.13.4-0.1.el6.noarch Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:1720 BZ<2>Jira Resync |
Created attachment 1439506 [details] /var/log/ from the engine Description of problem: With latest 3.6.13, can't reach the engine web APIs. server.log: 2018-05-21 12:15:17,210 INFO [org.quartz.core.JobRunShell] (DefaultQuartzScheduler_Worker-16) Job DEFAULT.org.ovirt.engine.core.bll.tasks.CommandExecutor.invokeCallbackMethods#-9223372036854775787 threw a JobExecutionException: : org.quartz.JobExecutionException: failed to execute job at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source) [:1.8.0_121] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_121] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_121] at org.ovirt.engine.core.utils.timer.JobWrapper.invokeMethod(JobWrapper.java:81) [scheduler.jar:] at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:52) [scheduler.jar:] at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] Version-Release number of selected component (if applicable): [root@jenkins-vm-03 ~]# rpm -q rhevm rhevm-3.6.13.1-0.1.el6.noarch [root@jenkins-vm-03 ~]# cat /etc/redhat-release Red Hat Enterprise Linux Server release 6.9 (Santiago) How reproducible: 1/1 Actual results: Web APIs don't function, we get 404 http error Additional info: httpd and ovirt-engine are running: [root@jenkins-vm-03 ~]# service httpd status httpd (pid 2077) is running... [root@jenkins-vm-03 ~]# service ovirt-engine status ovirt-engine (pid 13255) is running... The setup is alive, please contact me for its details