Bug 798914 - ovirt-engine-core : GSSAPI Authentication errors when using Rest API
ovirt-engine-core : GSSAPI Authentication errors when using Rest API
Status: CLOSED NOTABUG
Product: oVirt
Classification: Community
Component: ovirt-engine-core (Show other bugs)
unspecified
Unspecified Unspecified
urgent Severity high
: ---
: ---
Assigned To: Roy Golan
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-03-01 04:16 EST by Oded Ramraz
Modified: 2012-03-07 06:23 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-03-07 06:23:25 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
engine logs (83.41 KB, application/zip)
2012-03-01 04:16 EST, Oded Ramraz
no flags Details
kerberos debug messages (11.19 KB, text/plain)
2012-03-07 06:17 EST, Roy Golan
no flags Details

  None (edit)
Description Oded Ramraz 2012-03-01 04:16:13 EST
Description of problem:

Sometimes when running scripts using Rest API , few operations fails due to authentication errors ( see Additional Information )
We are using GSSAPI authentication in our environment. 

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

2012-02-29 22:12:00,160 ERROR [org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy] (http--0.0.0.0-8080-2) Error from Kerberos: java.lang.NullPointerException
	at java.io.ByteArrayInputStream.<init>(ByteArrayInputStream.java:106)
	at sun.security.util.DerValue.<init>(DerValue.java:294)
	at sun.security.krb5.KrbAsRep.<init>(KrbAsRep.java:54)
	at sun.security.krb5.KrbAsReq.getReply(KrbAsReq.java:469)
	at sun.security.krb5.Credentials.sendASRequest(Credentials.java:399)
	at sun.security.krb5.Credentials.acquireTGT(Credentials.java:348)
	at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:679)
	at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:559)
	at sun.reflect.GeneratedMethodAccessor70.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)
	at javax.security.auth.login.LoginContext.invoke(LoginContext.java:784)
	at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
	at javax.security.auth.login.LoginContext$4.run(LoginContext.java:698)
	at javax.security.auth.login.LoginContext$4.run(LoginContext.java:696)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:695)
	at javax.security.auth.login.LoginContext.login(LoginContext.java:594)
	at org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy.authenticateToKDC(GSSAPIDirContextAuthenticationStrategy.java:127)
	at org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy.explicitAuth(GSSAPIDirContextAuthenticationStrategy.java:119)
	at org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy.authenticate(GSSAPIDirContextAuthenticationStrategy.java:111)
	at org.ovirt.engine.core.bll.adbroker.GSSAPILdapTemplateWrapper.useAuthenticationStrategy(GSSAPILdapTemplateWrapper.java:85)
	at org.ovirt.engine.core.bll.adbroker.PrepareLdapConnectionTask.call(PrepareLdapConnectionTask.java:56)
	at org.ovirt.engine.core.bll.adbroker.DirectorySearcher.find(DirectorySearcher.java:98)
	at org.ovirt.engine.core.bll.adbroker.DirectorySearcher.FindOne(DirectorySearcher.java:39)
	at org.ovirt.engine.core.bll.adbroker.LdapAuthenticateUserCommand.executeQuery(LdapAuthenticateUserCommand.java:44)
	at org.ovirt.engine.core.bll.adbroker.LdapBrokerCommandBase.Execute(LdapBrokerCommandBase.java:68)
	at org.ovirt.engine.core.bll.adbroker.LdapBrokerBase.RunAdAction(LdapBrokerBase.java:18)
	at org.ovirt.engine.core.bll.LoginUserCommand.AuthenticateUser(LoginUserCommand.java:36)
	at org.ovirt.engine.core.bll.LoginBaseCommand.isUserCanBeAuthenticated(LoginBaseCommand.java:183)
	at org.ovirt.engine.core.bll.LoginAdminUserCommand.canDoAction(LoginAdminUserCommand.java:18)
	at org.ovirt.engine.core.bll.CommandBase.InternalCanDoAction(CommandBase.java:391)
	at org.ovirt.engine.core.bll.CommandBase.ExecuteAction(CommandBase.java:202)
	at org.ovirt.engine.core.bll.Backend.Login(Backend.java:470)
	at sun.reflect.GeneratedMethodAccessor68.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)
	at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72)
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
	at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374)
	at org.ovirt.engine.core.utils.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:11)
	at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)
	at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123)
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
	at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53)
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
	at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45)
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
	at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21)
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
	at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53)
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
	at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53)
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
	at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:221)
	at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:367)
	at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:205)
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
	at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
	at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:44)
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
	at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:55)
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
	at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165)
	at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173)
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
	at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72)
	at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view10.Login(Unknown Source)
	at sun.reflect.GeneratedMethodAccessor67.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:616)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:198)
	at $Proxy34.Login(Unknown Source)
	at org.ovirt.engine.api.restapi.security.auth.LoginValidator.validate(LoginValidator.java:66)
	at org.ovirt.engine.api.common.security.auth.Challenger.preProcess(Challenger.java:85)
	at org.jboss.resteasy.core.ResourceMethod.invokeOnTarget(ResourceMethod.java:245)
	at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:220)
	at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:209)
	at org.jboss.resteasy.core.SynchronousDispatcher.getResponse(SynchronousDispatcher.java:519)
	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:496)
	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:119)
	at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:208)
	at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:55)
	at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:50)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
	at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:151)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:362)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877)
	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671)
	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930)
	at java.lang.Thread.run(Thread.java:679)

2012-02-29 22:12:00,162 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (http--0.0.0.0-8080-2) Failed ldap search server ldap://qa1.qa.lab.tlv.redhat.com:389 due to org.ovirt.engine.core.bll.adbroker.EngineDirectoryServiceException. We should not try the next server: org.ovirt.engine.core.bll.adbroker.EngineDirectoryServiceException
	at org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy.authenticateToKDC(GSSAPIDirContextAuthenticationStrategy.java:150) [engine-bll.jar:]
	at org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy.explicitAuth(GSSAPIDirContextAuthenticationStrategy.java:119) [engine-bll.jar:]
	at org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy.authenticate(GSSAPIDirContextAuthenticationStrategy.java:111) [engine-bll.jar:]
	at org.ovirt.engine.core.bll.adbroker.GSSAPILdapTemplateWrapper.useAuthenticationStrategy(GSSAPILdapTemplateWrapper.java:85) [engine-bll.jar:]
	at org.ovirt.engine.core.bll.adbroker.PrepareLdapConnectionTask.call(PrepareLdapConnectionTask.java:56) [engine-bll.jar:]
	at org.ovirt.engine.core.bll.adbroker.DirectorySearcher.find(DirectorySearcher.java:98) [engine-bll.jar:]
	at org.ovirt.engine.core.bll.adbroker.DirectorySearcher.FindOne(DirectorySearcher.java:39) [engine-bll.jar:]
	at org.ovirt.engine.core.bll.adbroker.LdapAuthenticateUserCommand.executeQuery(LdapAuthenticateUserCommand.java:44) [engine-bll.jar:]
	at org.ovirt.engine.core.bll.adbroker.LdapBrokerCommandBase.Execute(LdapBrokerCommandBase.java:68) [engine-bll.jar:]
	at org.ovirt.engine.core.bll.adbroker.LdapBrokerBase.RunAdAction(LdapBrokerBase.java:18) [engine-bll.jar:]
	at org.ovirt.engine.core.bll.LoginUserCommand.AuthenticateUser(LoginUserCommand.java:36) [engine-bll.jar:]
	at org.ovirt.engine.core.bll.LoginBaseCommand.isUserCanBeAuthenticated(LoginBaseCommand.java:183) [engine-bll.jar:]
	at org.ovirt.engine.core.bll.LoginAdminUserCommand.canDoAction(LoginAdminUserCommand.java:18) [engine-bll.jar:]
	at org.ovirt.engine.core.bll.CommandBase.InternalCanDoAction(CommandBase.java:391) [engine-bll.jar:]
	at org.ovirt.engine.core.bll.CommandBase.ExecuteAction(CommandBase.java:202) [engine-bll.jar:]
	at org.ovirt.engine.core.bll.Backend.Login(Backend.java:470) [engine-bll.jar:]
	at sun.reflect.GeneratedMethodAccessor68.invoke(Unknown Source) [:1.6.0_24]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [:1.6.0_24]
	at java.lang.reflect.Method.invoke(Method.java:616) [:1.6.0_24]
	at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.0.Beta1b.jar:]
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:]
	at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:]
	at org.ovirt.engine.core.utils.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:11) [engine-utils.jar:]
	at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source) [:1.6.0_24]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [:1.6.0_24]
	at java.lang.reflect.Method.invoke(Method.java:616) [:1.6.0_24]
	at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.0.Beta1b.jar:]
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:]
	at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:]
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:]
	at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.0.Beta1b.jar:]
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:]
	at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:]
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:]
	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:]
	at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.0.Beta1b.jar:]
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:]
	at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.0.Beta1b.jar:]
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:]
	at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:221) [jboss-as-ejb3-7.1.0.Beta1b.jar:]
	at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:367) [jboss-as-ejb3-7.1.0.Beta1b.jar:]
	at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:205) [jboss-as-ejb3-7.1.0.Beta1b.jar:]
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:]
	at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.0.Beta1b.jar:]
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:]
	at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:44) [jboss-as-ee-7.1.0.Beta1b.jar:]
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:]
	at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:55) [jboss-as-ejb3-7.1.0.Beta1b.jar:]
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:]
	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:]
	at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.0.Beta1b.jar:]
	at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.0.Beta1b.jar:]
	at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:]
	at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:]
	at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.0.Beta1b.jar:]
	at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view10.Login(Unknown Source) [engine-common.jar:]
	at sun.reflect.GeneratedMethodAccessor67.invoke(Unknown Source) [:1.6.0_24]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [:1.6.0_24]
	at java.lang.reflect.Method.invoke(Method.java:616) [:1.6.0_24]
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307) [spring-aop-2.5.6.SEC02.jar:]
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:198) [spring-aop-2.5.6.SEC02.jar:]
	at $Proxy34.Login(Unknown Source)	at org.ovirt.engine.api.restapi.security.auth.LoginValidator.validate(LoginValidator.java:66)
	at org.ovirt.engine.api.common.security.auth.Challenger.preProcess(Challenger.java:85)
	at org.jboss.resteasy.core.ResourceMethod.invokeOnTarget(ResourceMethod.java:245) [resteasy-jaxrs-2.2.3.GA.jar:]
	at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:220) [resteasy-jaxrs-2.2.3.GA.jar:]
	at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:209) [resteasy-jaxrs-2.2.3.GA.jar:]
	at org.jboss.resteasy.core.SynchronousDispatcher.getResponse(SynchronousDispatcher.java:519) [resteasy-jaxrs-2.2.3.GA.jar:]
	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:496) [resteasy-jaxrs-2.2.3.GA.jar:]
	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:119) [resteasy-jaxrs-2.2.3.GA.jar:]
	at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:208) [resteasy-jaxrs-2.2.3.GA.jar:]
	at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:55) [resteasy-jaxrs-2.2.3.GA.jar:]
	at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:50) [resteasy-jaxrs-2.2.3.GA.jar:]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:]
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
	at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:151)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:362)
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877)
	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671)
	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930)
	at java.lang.Thread.run(Thread.java:679) [:1.6.0_24]
Comment 1 Oded Ramraz 2012-03-01 04:16:47 EST
Created attachment 566785 [details]
engine logs
Comment 2 lpeer 2012-03-05 06:12:10 EST
According to Oved this is a hard one to catch.
Do we have a way to reproduce this in a consistent way?
Comment 3 Oded Ramraz 2012-03-05 07:41:14 EST
We don't have a consistent way to reproduce it , but it happen from time to time. 
rgolan@redhat.com recommend me to open this issue in order to investigate it / add it to release notes.
Comment 4 Roy Golan 2012-03-07 06:17:44 EST
Created attachment 568259 [details]
kerberos debug messages
Comment 5 Roy Golan 2012-03-07 06:23:25 EST
I have debugged jboss with -Dsun.security.krb5.debug=true
Took like half an our of automation cycle for that to occur but we can't rely on that as a reporducer.

The debug messages clearly shows the krb is trying to read 0 bytes from the reply:

2012-03-06 17:28:26,873 INFO  [stdout] (http--0.0.0.0-8080-2) >>> KrbKdcReq send: kdc=qa1.qa.lab.tlv.redhat.com TCP:88, timeout=30000, number of retries =3, #bytes=1368
2012-03-06 17:28:26,874 INFO  [stdout] (http--0.0.0.0-8080-2) >>>DEBUG: TCPClient could not read length field
2012-03-06 17:28:26,874 INFO  [stdout] (http--0.0.0.0-8080-2) >>> KrbKdcReq send: #bytes read=0
2012-03-06 17:28:26,874 INFO  [stdout] (http--0.0.0.0-8080-2) >>> KrbKdcReq send: #bytes read=0

Following this the NPE occurs due to null input buffer.

qa1.qa eventlog have many errors on kerberos, master browser(what ever that means) , near the ocurance of the error.

I cant find the reason why the kerberos service have an empty reply or did it 
close the connection but anyhow our behaviour is to try the next server if exist which is good.

log is attached.

Note You need to log in before you can comment on or make changes to this bug.