Description of problem: Sometimes when running scripts using Rest API , few operations fails due to authentication errors ( see Additional Information ) We are using GSSAPI authentication in our environment. Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info: 2012-02-29 22:12:00,160 ERROR [org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy] (http--0.0.0.0-8080-2) Error from Kerberos: java.lang.NullPointerException at java.io.ByteArrayInputStream.<init>(ByteArrayInputStream.java:106) at sun.security.util.DerValue.<init>(DerValue.java:294) at sun.security.krb5.KrbAsRep.<init>(KrbAsRep.java:54) at sun.security.krb5.KrbAsReq.getReply(KrbAsReq.java:469) at sun.security.krb5.Credentials.sendASRequest(Credentials.java:399) at sun.security.krb5.Credentials.acquireTGT(Credentials.java:348) at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:679) at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:559) at sun.reflect.GeneratedMethodAccessor70.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at javax.security.auth.login.LoginContext.invoke(LoginContext.java:784) at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203) at javax.security.auth.login.LoginContext$4.run(LoginContext.java:698) at javax.security.auth.login.LoginContext$4.run(LoginContext.java:696) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:695) at javax.security.auth.login.LoginContext.login(LoginContext.java:594) at org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy.authenticateToKDC(GSSAPIDirContextAuthenticationStrategy.java:127) at org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy.explicitAuth(GSSAPIDirContextAuthenticationStrategy.java:119) at org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy.authenticate(GSSAPIDirContextAuthenticationStrategy.java:111) at org.ovirt.engine.core.bll.adbroker.GSSAPILdapTemplateWrapper.useAuthenticationStrategy(GSSAPILdapTemplateWrapper.java:85) at org.ovirt.engine.core.bll.adbroker.PrepareLdapConnectionTask.call(PrepareLdapConnectionTask.java:56) at org.ovirt.engine.core.bll.adbroker.DirectorySearcher.find(DirectorySearcher.java:98) at org.ovirt.engine.core.bll.adbroker.DirectorySearcher.FindOne(DirectorySearcher.java:39) at org.ovirt.engine.core.bll.adbroker.LdapAuthenticateUserCommand.executeQuery(LdapAuthenticateUserCommand.java:44) at org.ovirt.engine.core.bll.adbroker.LdapBrokerCommandBase.Execute(LdapBrokerCommandBase.java:68) at org.ovirt.engine.core.bll.adbroker.LdapBrokerBase.RunAdAction(LdapBrokerBase.java:18) at org.ovirt.engine.core.bll.LoginUserCommand.AuthenticateUser(LoginUserCommand.java:36) at org.ovirt.engine.core.bll.LoginBaseCommand.isUserCanBeAuthenticated(LoginBaseCommand.java:183) at org.ovirt.engine.core.bll.LoginAdminUserCommand.canDoAction(LoginAdminUserCommand.java:18) at org.ovirt.engine.core.bll.CommandBase.InternalCanDoAction(CommandBase.java:391) at org.ovirt.engine.core.bll.CommandBase.ExecuteAction(CommandBase.java:202) at org.ovirt.engine.core.bll.Backend.Login(Backend.java:470) at sun.reflect.GeneratedMethodAccessor68.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) at org.ovirt.engine.core.utils.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:11) at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:221) at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:367) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:205) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:44) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:55) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view10.Login(Unknown Source) at sun.reflect.GeneratedMethodAccessor67.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:198) at $Proxy34.Login(Unknown Source) at org.ovirt.engine.api.restapi.security.auth.LoginValidator.validate(LoginValidator.java:66) at org.ovirt.engine.api.common.security.auth.Challenger.preProcess(Challenger.java:85) at org.jboss.resteasy.core.ResourceMethod.invokeOnTarget(ResourceMethod.java:245) at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:220) at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:209) at org.jboss.resteasy.core.SynchronousDispatcher.getResponse(SynchronousDispatcher.java:519) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:496) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:119) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:208) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:55) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:50) at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:151) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:362) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) at java.lang.Thread.run(Thread.java:679) 2012-02-29 22:12:00,162 ERROR [org.ovirt.engine.core.bll.adbroker.DirectorySearcher] (http--0.0.0.0-8080-2) Failed ldap search server ldap://qa1.qa.lab.tlv.redhat.com:389 due to org.ovirt.engine.core.bll.adbroker.EngineDirectoryServiceException. We should not try the next server: org.ovirt.engine.core.bll.adbroker.EngineDirectoryServiceException at org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy.authenticateToKDC(GSSAPIDirContextAuthenticationStrategy.java:150) [engine-bll.jar:] at org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy.explicitAuth(GSSAPIDirContextAuthenticationStrategy.java:119) [engine-bll.jar:] at org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy.authenticate(GSSAPIDirContextAuthenticationStrategy.java:111) [engine-bll.jar:] at org.ovirt.engine.core.bll.adbroker.GSSAPILdapTemplateWrapper.useAuthenticationStrategy(GSSAPILdapTemplateWrapper.java:85) [engine-bll.jar:] at org.ovirt.engine.core.bll.adbroker.PrepareLdapConnectionTask.call(PrepareLdapConnectionTask.java:56) [engine-bll.jar:] at org.ovirt.engine.core.bll.adbroker.DirectorySearcher.find(DirectorySearcher.java:98) [engine-bll.jar:] at org.ovirt.engine.core.bll.adbroker.DirectorySearcher.FindOne(DirectorySearcher.java:39) [engine-bll.jar:] at org.ovirt.engine.core.bll.adbroker.LdapAuthenticateUserCommand.executeQuery(LdapAuthenticateUserCommand.java:44) [engine-bll.jar:] at org.ovirt.engine.core.bll.adbroker.LdapBrokerCommandBase.Execute(LdapBrokerCommandBase.java:68) [engine-bll.jar:] at org.ovirt.engine.core.bll.adbroker.LdapBrokerBase.RunAdAction(LdapBrokerBase.java:18) [engine-bll.jar:] at org.ovirt.engine.core.bll.LoginUserCommand.AuthenticateUser(LoginUserCommand.java:36) [engine-bll.jar:] at org.ovirt.engine.core.bll.LoginBaseCommand.isUserCanBeAuthenticated(LoginBaseCommand.java:183) [engine-bll.jar:] at org.ovirt.engine.core.bll.LoginAdminUserCommand.canDoAction(LoginAdminUserCommand.java:18) [engine-bll.jar:] at org.ovirt.engine.core.bll.CommandBase.InternalCanDoAction(CommandBase.java:391) [engine-bll.jar:] at org.ovirt.engine.core.bll.CommandBase.ExecuteAction(CommandBase.java:202) [engine-bll.jar:] at org.ovirt.engine.core.bll.Backend.Login(Backend.java:470) [engine-bll.jar:] at sun.reflect.GeneratedMethodAccessor68.invoke(Unknown Source) [:1.6.0_24] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [:1.6.0_24] at java.lang.reflect.Method.invoke(Method.java:616) [:1.6.0_24] at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.0.Beta1b.jar:] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:] at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation-1.1.1.Final.jar:] at org.ovirt.engine.core.utils.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:11) [engine-utils.jar:] at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source) [:1.6.0_24] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [:1.6.0_24] at java.lang.reflect.Method.invoke(Method.java:616) [:1.6.0_24] at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.0.Beta1b.jar:] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:] at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.1.Final.jar:] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:] at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.1.0.Beta1b.jar:] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:] at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.1.Final.jar:] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:] at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:] at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.0.Beta1b.jar:] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:] at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [jboss-as-ejb3-7.1.0.Beta1b.jar:] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:] at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:221) [jboss-as-ejb3-7.1.0.Beta1b.jar:] at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:367) [jboss-as-ejb3-7.1.0.Beta1b.jar:] at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:205) [jboss-as-ejb3-7.1.0.Beta1b.jar:] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:] at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.1.0.Beta1b.jar:] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:] at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:44) [jboss-as-ee-7.1.0.Beta1b.jar:] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:] at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:55) [jboss-as-ejb3-7.1.0.Beta1b.jar:] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:] at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:] at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.1.0.Beta1b.jar:] at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173) [jboss-as-ee-7.1.0.Beta1b.jar:] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.1.Final.jar:] at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.1.Final.jar:] at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.1.0.Beta1b.jar:] at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view10.Login(Unknown Source) [engine-common.jar:] at sun.reflect.GeneratedMethodAccessor67.invoke(Unknown Source) [:1.6.0_24] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [:1.6.0_24] at java.lang.reflect.Method.invoke(Method.java:616) [:1.6.0_24] at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307) [spring-aop-2.5.6.SEC02.jar:] at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:198) [spring-aop-2.5.6.SEC02.jar:] at $Proxy34.Login(Unknown Source) at org.ovirt.engine.api.restapi.security.auth.LoginValidator.validate(LoginValidator.java:66) at org.ovirt.engine.api.common.security.auth.Challenger.preProcess(Challenger.java:85) at org.jboss.resteasy.core.ResourceMethod.invokeOnTarget(ResourceMethod.java:245) [resteasy-jaxrs-2.2.3.GA.jar:] at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:220) [resteasy-jaxrs-2.2.3.GA.jar:] at org.jboss.resteasy.core.ResourceMethod.invoke(ResourceMethod.java:209) [resteasy-jaxrs-2.2.3.GA.jar:] at org.jboss.resteasy.core.SynchronousDispatcher.getResponse(SynchronousDispatcher.java:519) [resteasy-jaxrs-2.2.3.GA.jar:] at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:496) [resteasy-jaxrs-2.2.3.GA.jar:] at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:119) [resteasy-jaxrs-2.2.3.GA.jar:] at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:208) [resteasy-jaxrs-2.2.3.GA.jar:] at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:55) [resteasy-jaxrs-2.2.3.GA.jar:] at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:50) [resteasy-jaxrs-2.2.3.GA.jar:] at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:151) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:362) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) at java.lang.Thread.run(Thread.java:679) [:1.6.0_24]
Created attachment 566785 [details] engine logs
According to Oved this is a hard one to catch. Do we have a way to reproduce this in a consistent way?
We don't have a consistent way to reproduce it , but it happen from time to time. rgolan recommend me to open this issue in order to investigate it / add it to release notes.
Created attachment 568259 [details] kerberos debug messages
I have debugged jboss with -Dsun.security.krb5.debug=true Took like half an our of automation cycle for that to occur but we can't rely on that as a reporducer. The debug messages clearly shows the krb is trying to read 0 bytes from the reply: 2012-03-06 17:28:26,873 INFO [stdout] (http--0.0.0.0-8080-2) >>> KrbKdcReq send: kdc=qa1.qa.lab.tlv.redhat.com TCP:88, timeout=30000, number of retries =3, #bytes=1368 2012-03-06 17:28:26,874 INFO [stdout] (http--0.0.0.0-8080-2) >>>DEBUG: TCPClient could not read length field 2012-03-06 17:28:26,874 INFO [stdout] (http--0.0.0.0-8080-2) >>> KrbKdcReq send: #bytes read=0 2012-03-06 17:28:26,874 INFO [stdout] (http--0.0.0.0-8080-2) >>> KrbKdcReq send: #bytes read=0 Following this the NPE occurs due to null input buffer. qa1.qa eventlog have many errors on kerberos, master browser(what ever that means) , near the ocurance of the error. I cant find the reason why the kerberos service have an empty reply or did it close the connection but anyhow our behaviour is to try the next server if exist which is good. log is attached.