Bug 1580398 - [LP - RHV product] (6.4.x) Upgrade to jackson-1.9.9-12 in EAP 6.4.20.CR1 breaks RHV
Summary: [LP - RHV product] (6.4.x) Upgrade to jackson-1.9.9-12 in EAP 6.4.20.CR1 brea...
Keywords:
Status: VERIFIED
Alias: None
Product: JBoss Enterprise Application Platform 6
Classification: JBoss
Component: Server
Version: 6.4.20
Hardware: x86_64
OS: Unspecified
urgent
urgent
Target Milestone: CR1
: EAP 6.4.21
Assignee: Jiri Ondrusek
QA Contact: Peter Mackay
eap-docs
URL:
Whiteboard:
Depends On:
Blocks: eap6421-payload 1577407 1580319 1614448 1623544
TreeView+ depends on / blocked
 
Reported: 2018-05-21 12:22 UTC by Vaclav Tunka
Modified: 2019-05-31 22:02 UTC (History)
14 users (show)

Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1580319
Environment:
Last Closed:
Type: Bug


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 3442891 None None None 2018-08-07 15:28:26 UTC
JBoss Issue Tracker JBEAP-14760 Blocker Closed [LP - RHV product] (7.1.z) Upgrade to jackson-1.9.9-12 in EAP 6.4.20.CR1 breaks RHV 2019-05-17 06:38:29 UTC
Red Hat Knowledge Base (Solution) 3452511 None None None 2018-05-30 18:15:00 UTC

Description Vaclav Tunka 2018-05-21 12:22:08 UTC
+++ This bug was initially created as a clone of Bug #1580319 +++

Description of problem:
With latest 3.6.13, can't reach the engine web APIs. server.log:

2018-05-21 12:15:17,210 INFO  [org.quartz.core.JobRunShell] (DefaultQuartzScheduler_Worker-16) Job DEFAULT.org.ovirt.engine.core.bll.tasks.CommandExecutor.invokeCallbackMethods#-9223372036854775787 threw a JobExecutionException: : org.quartz.JobExecutionException: failed to execute job
        at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source) [:1.8.0_121]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_121]
        at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_121]
        at org.ovirt.engine.core.utils.timer.JobWrapper.invokeMethod(JobWrapper.java:81) [scheduler.jar:]
        at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:52) [scheduler.jar:]
        at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:]
        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:]


Version-Release number of selected component (if applicable):
[root@jenkins-vm-03 ~]# rpm -q rhevm
rhevm-3.6.13.1-0.1.el6.noarch
[root@jenkins-vm-03 ~]# cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 6.9 (Santiago)



How reproducible:
1/1


Actual results:
Web APIs don't function, we get 404 http error



Additional info:
httpd and ovirt-engine are running:

[root@jenkins-vm-03 ~]# service httpd status
httpd (pid  2077) is running...
[root@jenkins-vm-03 ~]# service ovirt-engine status
ovirt-engine (pid  13255) is running...


The setup is alive, please contact me for its details

--- Additional comment from Martin Perina on 2018-05-21 06:35:25 EDT ---

Below error started the issue:

2018-05-19 23:26:31,255 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 51) MSC000001: Failed to start service jboss.deployment.subunit."engine.ear"."bll.jar".component.InitBackendServicesOnStartupBean.START: org.jboss.msc.service.StartException in service jboss.deployment.subunit."engine.ear"."bll.jar".component.InitBackendServicesOnStartupBean.START: java.lang.IllegalStateException: JBAS011048: Failed to construct component instance
        at org.jboss.as.ee.component.ComponentStartService$1.run(ComponentStartService.java:57) [jboss-as-ee.jar:7.5.20.Final-redhat-1]                                                                       
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_121]                                                                                                         
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) [rt.jar:1.8.0_121]                                                                                                                        
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_121]                                                                                                 
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_121]                                                                                                 
        at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_121]                                                                                                                                           
        at org.jboss.threads.JBossThread.run(JBossThread.java:122)                                                                                                                                            
Caused by: java.lang.IllegalStateException: JBAS011048: Failed to construct component instance                                                                                                                
        at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:164) [jboss-as-ee.jar:7.5.20.Final-redhat-1]                                                               
        at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:135) [jboss-as-ee.jar:7.5.20.Final-redhat-1]                                                               
        at org.jboss.as.ee.component.BasicComponent.createInstance(BasicComponent.java:90) [jboss-as-ee.jar:7.5.20.Final-redhat-1]                                                                            
        at org.jboss.as.ejb3.component.singleton.SingletonComponent.getComponentInstance(SingletonComponent.java:122)                                                                                         
        at org.jboss.as.ejb3.component.singleton.SingletonComponent.start(SingletonComponent.java:137)                                                                                                        
        at org.jboss.as.ee.component.ComponentStartService$1.run(ComponentStartService.java:54) [jboss-as-ee.jar:7.5.20.Final-redhat-1]                                                                       
        ... 6 more                                                                                                                                                                                            
Caused by: javax.ejb.EJBException: org.apache.commons.lang.SerializationException: org.codehaus.jackson.map.JsonMappingException: Illegal type (org.ovirt.engine.core.common.action.AddImageFromScratchParameters) to deserialize: prevented for security reasons                                                                                                                    
        at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleExceptionInOurTx(CMTTxInterceptor.java:191)                                                                                                            
        at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:281)                                                                                                                     
        at org.jboss.as.ejb3.tx.CMTTxInterceptor.requiresNew(CMTTxInterceptor.java:353)                                                                                                                       
        at org.jboss.as.ejb3.tx.LifecycleCMTTxInterceptor.processInvocation(LifecycleCMTTxInterceptor.java:67)                                                                                                
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]                                                                           
        at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)                                                        
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]                                                                           
        at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:70) [jboss-invocation.jar:1.1.3.Final-redhat-1]                                            
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]                                                                           
        at org.jboss.as.ejb3.component.singleton.StartupCountDownInterceptor.processInvocation(StartupCountDownInterceptor.java:25)                                                                           
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]                                                                           
        at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.3.Final-redhat-1]                                                                  
        at org.jboss.as.ee.component.BasicComponent.constructComponentInstance(BasicComponent.java:162) [jboss-as-ee.jar:7.5.20.Final-redhat-1]                                                               
        ... 11 more                                                                                                                                                                                           
Caused by: org.apache.commons.lang.SerializationException: org.codehaus.jackson.map.JsonMappingException: Illegal type (org.ovirt.engine.core.common.action.AddImageFromScratchParameters) to deserialize: prevented for security reasons                                                                                                                                            
        at org.ovirt.engine.core.utils.serialization.json.JsonObjectDeserializer.readJsonString(JsonObjectDeserializer.java:101)                                                                              
        at org.ovirt.engine.core.utils.serialization.json.JsonObjectDeserializer.deserialize(JsonObjectDeserializer.java:57)                                                                                  
        at org.ovirt.engine.core.dao.CommandEntityDaoImpl.deserializeParameters(CommandEntityDaoImpl.java:150)                                                                                                
        at org.ovirt.engine.core.dao.CommandEntityDaoImpl.access$000(CommandEntityDaoImpl.java:32)                                                                                                            
        at org.ovirt.engine.core.dao.CommandEntityDaoImpl$1.mapRow(CommandEntityDaoImpl.java:71)                                                                                                              
        at org.ovirt.engine.core.dao.CommandEntityDaoImpl$1.mapRow(CommandEntityDaoImpl.java:58)                                                                                                              
        at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:92)                                                                                         
        at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:1)                                                                                          
        at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:649)                                                                                                          
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587)                                                                                                                          
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637)                                                                                                                            
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666)                                                                                                                            
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706)                                                                                                                            
        at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154)                                                            
        at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120)                                                                      
        at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181)                                                                                                               
        at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147)                                                                                             
        at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:109)                                                                                         
        at org.ovirt.engine.core.dao.DefaultReadDao.getAll(DefaultReadDao.java:77)                                                                                                                            
        at org.ovirt.engine.core.bll.tasks.CommandsCacheImpl.initializeCache(CommandsCacheImpl.java:33)                                                                                                       
        at org.ovirt.engine.core.bll.tasks.CommandsCacheImpl.get(CommandsCacheImpl.java:50)                                                                                                                   
        at org.ovirt.engine.core.bll.tasks.CommandCoordinatorImpl.getCommandEntity(CommandCoordinatorImpl.java:132)                                                                                           
        at org.ovirt.engine.core.bll.tasks.CoCoAsyncTaskHelper.getCommandEntity(CoCoAsyncTaskHelper.java:160)                                                                                                 
        at org.ovirt.engine.core.bll.tasks.CoCoAsyncTaskHelper.getAllAsyncTasksFromDb(CoCoAsyncTaskHelper.java:153)                                                                                           
        at org.ovirt.engine.core.bll.tasks.CommandCoordinatorImpl.getAllAsyncTasksFromDb(CommandCoordinatorImpl.java:292)                                                                                     
        at org.ovirt.engine.core.bll.tasks.AsyncTaskManager.initAsyncTaskManager(AsyncTaskManager.java:113)                                                                                                   
        at org.ovirt.engine.core.bll.tasks.CommandCoordinatorUtil.initAsyncTaskManager(CommandCoordinatorUtil.java:49)                                                                                        
        at org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean.create(InitBackendServicesOnStartupBean.java:60)                                                                                        
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_121]                                                                                                                     
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_121]                                                                                                   
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_121]                                                                                           
        at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_121]                                                                                                                                
        at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:96) [jboss-as-ee.jar:7.5.20.Final-redhat-1]                 
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]                                                                           
        at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.3.Final-redhat-1]                                                                    
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]                                                                           
        at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doLifecycleInterception(Jsr299BindingsInterceptor.java:116)                                                                                        
        at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:107)                                                                                              
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]                                                                           
        at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:93)                                                                         
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]                                                                           
        at org.jboss.as.weld.injection.WeldInjectionInterceptor.processInvocation(WeldInjectionInterceptor.java:67)                                                                                           
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]                                                                           
        at org.jboss.as.ee.component.ComponentInstantiatorInterceptor.processInvocation(ComponentInstantiatorInterceptor.java:76) [jboss-as-ee.jar:7.5.20.Final-redhat-1]                                     
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]                                                                           
        at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.3.Final-redhat-1]                                                                    
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]                                                                           
        at org.jboss.as.weld.ejb.Jsr299BindingsCreateInterceptor.processInvocation(Jsr299BindingsCreateInterceptor.java:92)                                                                                   
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]                                                                           
        at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.5.20.Final-redhat-1]
        at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.3.Final-redhat-1]                                                                           
        at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279)                                                                                                                     
        ... 22 more                                                                                                                                                                                           
Caused by: org.codehaus.jackson.map.JsonMappingException: Illegal type (org.ovirt.engine.core.common.action.AddImageFromScratchParameters) to deserialize: prevented for security reasons                     
        at org.codehaus.jackson.map.deser.BeanDeserializerFactory.checkLegalTypes(BeanDeserializerFactory.java:1521)                                                                                          
        at org.codehaus.jackson.map.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:655)                                                                                    
        at org.codehaus.jackson.map.deser.StdDeserializerProvider._createDeserializer(StdDeserializerProvider.java:401)                                                                                       
        at org.codehaus.jackson.map.deser.StdDeserializerProvider._createAndCache2(StdDeserializerProvider.java:310)                                                                                          
        at org.codehaus.jackson.map.deser.StdDeserializerProvider._createAndCacheValueDeserializer(StdDeserializerProvider.java:290)                                                                          
        at org.codehaus.jackson.map.deser.StdDeserializerProvider.findValueDeserializer(StdDeserializerProvider.java:159)                                                                                     
        at org.codehaus.jackson.map.deser.StdDeserializerProvider.findTypedValueDeserializer(StdDeserializerProvider.java:180)                                                                                
        at org.codehaus.jackson.map.ObjectMapper._findRootDeserializer(ObjectMapper.java:2829)                                                                                                                
        at org.codehaus.jackson.map.ObjectMapper._readMapAndClose(ObjectMapper.java:2728)                                                                                                                     
        at org.codehaus.jackson.map.ObjectMapper.readValue(ObjectMapper.java:1863)                                                                                                                            
        at org.ovirt.engine.core.utils.serialization.json.JsonObjectDeserializer.readJsonString(JsonObjectDeserializer.java:99)                                                                               
        ... 73 more                                                                                                                                                                                           
                                                                                                                                                                                                              
2018-05-19 23:26:31,266 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) JBAS014612: Operation ("deploy") failed - address: ([("deployment" => "engine.ear")]) - failure description: {"JBAS014671: Failed services" => {"jboss.deployment.subunit.\"engine.ear\".\"bll.jar\".component.InitBackendServicesOnStartupBean.START" => "org.jboss.msc.service.StartException in service jboss.deployment.subunit.\"engine.ear\".\"bll.jar\".component.InitBackendServicesOnStartupBean.START: java.lang.IllegalStateException: JBAS011048: Failed to construct component instance
    Caused by: java.lang.IllegalStateException: JBAS011048: Failed to construct component instance                                                                                                            
    Caused by: javax.ejb.EJBException: org.apache.commons.lang.SerializationException: org.codehaus.jackson.map.JsonMappingException: Illegal type (org.ovirt.engine.core.common.action.AddImageFromScratchParameters) to deserialize: prevented for security reasons                                                                                                                
    Caused by: org.apache.commons.lang.SerializationException: org.codehaus.jackson.map.JsonMappingException: Illegal type (org.ovirt.engine.core.common.action.AddImageFromScratchParameters) to deserialize: prevented for security reasons                                                                                                                                        
    Caused by: org.codehaus.jackson.map.JsonMappingException: Illegal type (org.ovirt.engine.core.common.action.AddImageFromScratchParameters) to deserialize: prevented for security reasons"}}              


This is similar error as seen in BZ1577407, could you please try with EAP 6.4.19?

--- Additional comment from Martin Perina on 2018-05-21 07:00:28 EDT ---

I've just verified that downgrading jackson library included in EAP 6.4.19 fixed the issue, I will open EAP bug and attach it to this one and BZ1577407

--- Additional comment from Martin Perina on 2018-05-21 07:24:23 EDT ---

Workaround: use EAP 6.4.19 from http://download-ipv4.eng.brq.redhat.com/released/jboss/eap6/6.4.19/composes/JBEAP-6.4.19-RHEL-7/Server/x86_64/os/


Note You need to log in before you can comment on or make changes to this bug.