Help Desk Ticket Reference: https://c.na7.visual.force.com/apex/Case_View?id=500A000000BcqbUIAR&sfdc.override=1
Steps to Reproduce: Use either AD or another set of LDAP servers.
I used 2 instances of the Sun DS 7 server.

- a simple formlogin.war was attached to the JIRA. Name your security domain "LdapRealm" and have a user in a group called "JBossAdmin" (or change web.xml/jboss-web.xml). The context is /fl

- create a user in each LDAP server
- setup a LdapExtended login-module as per usual
  and add
      <module-option name="throwValidateError" value="true"/>
  then verify subsequently that authenticating to server 1 (user 1) and server 2 (user 2) works

- add a logger:
            <logger category="org.jboss.security">
                <level name="TRACE"/>
            </logger>

- create a dynamic referral from server 1 to server 2
  and configure the login-module to point to server 1

- Add the option
  <module-option name="java.naming.referral" value="ignore"/>

- log in with user 1 => works fine

- change the option to 
  <module-option name="java.naming.referral" value="follow"/>

  This will fail with the exceptions as described
  From the logs we can see that user 1 was logged in perfectly fine, but after that the referral was trying to be followed... thereby breaking the whole login.







project_key: JBPAPP6

We connect to AD with LdapExtLoginModule. It so happens that AD keeps  references to some external trees (such as "DomainDnsZones" and "ForestDnsZones") in the root of the LDAP tree. So when you configure LdapExtLoginModule to search any root, it will hit these referrals.

This normally fails with a standard 
{code}
javax.security.auth.login.FailedLoginException: Password Incorrect/Password Required
{code}

. This is not the whole story, though. If you enable the module option
{code}<module-option name="throwValidateError" value="true"/>{code}
, you get a more complete stack trace:

{code}
09:18:14,724 ERROR [org.jboss.security.authentication.JBossCachedAuthenticationManager] (http--127.0.0.1-8080-2) Login failure: javax.security.auth.login.FailedLoginException: Password Incorrect/Password Required
	at org.jboss.security.auth.spi.UsernamePasswordLoginModule.login(UsernamePasswordLoginModule.java:270) [picketbox-4.0.7.Final.jar:4.0.7.Final]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0]
	at java.lang.reflect.Method.invoke(Method.java:601) [rt.jar:1.7.0]
	at javax.security.auth.login.LoginContext.invoke(LoginContext.java:784) [rt.jar:1.7.0]
	at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203) [rt.jar:1.7.0]
	at javax.security.auth.login.LoginContext$4.run(LoginContext.java:698) [rt.jar:1.7.0]
	at javax.security.auth.login.LoginContext$4.run(LoginContext.java:696) [rt.jar:1.7.0]
	at java.security.AccessController.doPrivileged(Native Method) [rt.jar:1.7.0]
	at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:695) [rt.jar:1.7.0]
	at javax.security.auth.login.LoginContext.login(LoginContext.java:594) [rt.jar:1.7.0]
	at org.jboss.security.authentication.JBossCachedAuthenticationManager.defaultLogin(JBossCachedAuthenticationManager.java:449) [picketbox-infinispan-4.0.7.Final.jar:4.0.7.Final]
	at org.jboss.security.authentication.JBossCachedAuthenticationManager.proceedWithJaasLogin(JBossCachedAuthenticationManager.java:383) [picketbox-infinispan-4.0.7.Final.jar:4.0.7.Final]
	at org.jboss.security.authentication.JBossCachedAuthenticationManager.authenticate(JBossCachedAuthenticationManager.java:371) [picketbox-infinispan-4.0.7.Final.jar:4.0.7.Final]
	at org.jboss.security.authentication.JBossCachedAuthenticationManager.isValid(JBossCachedAuthenticationManager.java:160) [picketbox-infinispan-4.0.7.Final.jar:4.0.7.Final]
	at org.jboss.as.web.security.JBossWebRealm.authenticate(JBossWebRealm.java:214) [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]
	at org.apache.catalina.authenticator.FormAuthenticator.authenticate(FormAuthenticator.java:280) [jbossweb-7.0.13.Final.jar:]
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:381) [jbossweb-7.0.13.Final.jar:]
	at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.1.1.Final.jar:7.1.1.Final]
	at com.company.product.web.fix.ContextClassLoaderValve.invoke(ContextClassLoaderValve.java:19) [classes:]
	at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) [jbossweb-7.0.13.Final.jar:]
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) [jbossweb-7.0.13.Final.jar:]
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) [jbossweb-7.0.13.Final.jar:]
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) [jbossweb-7.0.13.Final.jar:]
	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) [jbossweb-7.0.13.Final.jar:]
	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) [jbossweb-7.0.13.Final.jar:]
	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) [jbossweb-7.0.13.Final.jar:]
	at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0]
Caused by: javax.naming.PartialResultException [Root exception is javax.naming.NotContextException: Cannot create context for: ldap://DomainDnsZones.global.scd.company.com/DC=DomainDnsZones,DC=global,DC=scd,DC=company,DC=com; remaining name 'dc=global,dc=scd,dc=company,dc=com']
	at com.sun.jndi.ldap.LdapNamingEnumeration.hasMoreImpl(LdapNamingEnumeration.java:242) [rt.jar:1.7.0]
	at com.sun.jndi.ldap.LdapNamingEnumeration.hasMore(LdapNamingEnumeration.java:189) [rt.jar:1.7.0]
	at org.jboss.security.auth.spi.LdapExtLoginModule.rolesSearch(LdapExtLoginModule.java:534) [picketbox-4.0.7.Final.jar:4.0.7.Final]
	at org.jboss.security.auth.spi.LdapExtLoginModule.createLdapInitContext(LdapExtLoginModule.java:445) [picketbox-4.0.7.Final.jar:4.0.7.Final]
	at org.jboss.security.auth.spi.LdapExtLoginModule.validatePassword(LdapExtLoginModule.java:312) [picketbox-4.0.7.Final.jar:4.0.7.Final]
	at org.jboss.security.auth.spi.UsernamePasswordLoginModule.login(UsernamePasswordLoginModule.java:267) [picketbox-4.0.7.Final.jar:4.0.7.Final]
	... 29 more
Caused by: javax.naming.NotContextException: Cannot create context for: ldap://DomainDnsZones.global.scd.company.com/DC=DomainDnsZones,DC=global,DC=scd,DC=company,DC=com; remaining name 'dc=global,dc=scd,dc=company,dc=com'
	at com.sun.jndi.ldap.LdapReferralContext.<init>(LdapReferralContext.java:141) [rt.jar:1.7.0]
	at com.sun.jndi.ldap.LdapReferralException.getReferralContext(LdapReferralException.java:150) [rt.jar:1.7.0]
	at com.sun.jndi.ldap.LdapNamingEnumeration.hasMoreReferrals(LdapNamingEnumeration.java:357) [rt.jar:1.7.0]
	at com.sun.jndi.ldap.LdapNamingEnumeration.hasMoreImpl(LdapNamingEnumeration.java:226) [rt.jar:1.7.0]
	... 34 more
{code}

When debugging this error, I concluded that the culprit is that ObjectFactoryBuilder doesn't resolve the reference correctly. getObjectInstance returns the reference instead of resolving it at the following location:

{code}
at org.jboss.as.naming.context.ObjectFactoryBuilder.getObjectInstance(ObjectFactoryBuilder.java:87)
	  at javax.naming.spi.NamingManager.getObjectInstance(NamingManager.java:300)
	  at com.sun.jndi.ldap.LdapReferralContext.<init>(LdapReferralContext.java:111)
	  at com.sun.jndi.ldap.LdapReferralException.getReferralContext(LdapReferralException.java:150)
	  at com.sun.jndi.ldap.LdapNamingEnumeration.hasMoreReferrals(LdapNamingEnumeration.java:357)
	  at com.sun.jndi.ldap.LdapNamingEnumeration.hasMoreImpl(LdapNamingEnumeration.java:226)
	  at com.sun.jndi.ldap.LdapNamingEnumeration.hasMore(LdapNamingEnumeration.java:189)
	  at org.jboss.security.auth.spi.LdapExtLoginModule.rolesSearch(LdapExtLoginModule.java:534)
	  at org.jboss.security.auth.spi.LdapExtLoginModule.createLdapInitContext(LdapExtLoginModule.java:445)
	  at org.jboss.security.auth.spi.LdapExtLoginModule.validatePassword(LdapExtLoginModule.java:312)
	  at org.jboss.security.auth.spi.UsernamePasswordLoginModule.login(UsernamePasswordLoginModule.java:267)
	  at sun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java:-1)
	  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	  at java.lang.reflect.Method.invoke(Method.java:601)
	  at javax.security.auth.login.LoginContext.invoke(LoginContext.java:784)
	  at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
	  at javax.security.auth.login.LoginContext$4.run(LoginContext.java:698)
	  at javax.security.auth.login.LoginContext$4.run(LoginContext.java:696)
	  at java.security.AccessController.doPrivileged(AccessController.java:-1)
	  at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:695)
	  at javax.security.auth.login.LoginContext.login(LoginContext.java:594)
	  at org.jboss.security.authentication.JBossCachedAuthenticationManager.defaultLogin(JBossCachedAuthenticationManager.java:449)
	  at org.jboss.security.authentication.JBossCachedAuthenticationManager.proceedWithJaasLogin(JBossCachedAuthenticationManager.java:383)
	  at org.jboss.security.authentication.JBossCachedAuthenticationManager.authenticate(JBossCachedAuthenticationManager.java:371)
	  at org.jboss.security.authentication.JBossCachedAuthenticationManager.isValid(JBossCachedAuthenticationManager.java:160)
	  at org.jboss.as.web.security.JBossWebRealm.authenticate(JBossWebRealm.java:214)
	  at org.apache.catalina.authenticator.FormAuthenticator.authenticate(FormAuthenticator.java:280)
	  at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:381)
	  at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50)
	  at com.company.product.web.fix.ContextClassLoaderValve.invoke(ContextClassLoaderValve.java:19)
	  at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153)
	  at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155)
	  at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
	  at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
	  at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368)
	  at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877)
	  at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671)
	  at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930)
	  at java.lang.Thread.run(Thread.java:722)
{code}

The relevant bit of code is:
{code}
 public Object getObjectInstance(final Object ref, final Name name, final Context nameCtx, final Hashtable<?, ?> environment) throws Exception {
        final ClassLoader classLoader = SecurityActions.getContextClassLoader();
        if(classLoader == null) {
            return ref;
        }
{code}

So this bit of code doesn't resolve the ref it the context classloader is null. Instead of aborting, it returns the ref unresolved. LdapReferralContext gets very confused when NamingManager doesn't resolve the reference, and throws the aforementioned NotContextException.

When debugging where the context classloader is set to null I found the following location:
{code}
http--127.0.0.1-8080-2@12911 daemon, prio=5, in group 'main', status: 'RUNNING'
	  at java.lang.Thread.setContextClassLoader(Thread.java:1480)
	  at org.jboss.security.auth.spi.SecurityActions$2.run(SecurityActions.java:59)
	  at org.jboss.security.auth.spi.SecurityActions$2.run(SecurityActions.java:56)
	  at java.security.AccessController.doPrivileged(AccessController.java:-1)
	  at org.jboss.security.auth.spi.SecurityActions.setContextClassLoader(SecurityActions.java:55)
	  at org.jboss.security.auth.spi.LdapExtLoginModule.createLdapInitContext(LdapExtLoginModule.java:435)
	  at org.jboss.security.auth.spi.LdapExtLoginModule.validatePassword(LdapExtLoginModule.java:312)
	  at org.jboss.security.auth.spi.UsernamePasswordLoginModule.login(UsernamePasswordLoginModule.java:267)
	  at sun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java:-1)
	  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	  at java.lang.reflect.Method.invoke(Method.java:601)
	  at javax.security.auth.login.LoginContext.invoke(LoginContext.java:784)
	  at javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
	  at javax.security.auth.login.LoginContext$4.run(LoginContext.java:698)
	  at javax.security.auth.login.LoginContext$4.run(LoginContext.java:696)
	  at java.security.AccessController.doPrivileged(AccessController.java:-1)
	  at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:695)
	  at javax.security.auth.login.LoginContext.login(LoginContext.java:594)
	  at org.jboss.security.authentication.JBossCachedAuthenticationManager.defaultLogin(JBossCachedAuthenticationManager.java:449)
	  at org.jboss.security.authentication.JBossCachedAuthenticationManager.proceedWithJaasLogin(JBossCachedAuthenticationManager.java:383)
	  at org.jboss.security.authentication.JBossCachedAuthenticationManager.authenticate(JBossCachedAuthenticationManager.java:371)
	  at org.jboss.security.authentication.JBossCachedAuthenticationManager.isValid(JBossCachedAuthenticationManager.java:160)
	  at org.jboss.as.web.security.JBossWebRealm.authenticate(JBossWebRealm.java:214)
	  at org.apache.catalina.authenticator.FormAuthenticator.authenticate(FormAuthenticator.java:280)
	  at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:381)
	  at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50)
	  at com.company.product.web.fix.ContextClassLoaderValve.invoke(ContextClassLoaderValve.java:19)
	  at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153)
	  at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155)
	  at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
	  at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
	  at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368)
	  at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877)
	  at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671)
	  at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930)
	  at java.lang.Thread.run(Thread.java:722)
{code}

Unfortunately I haven't been able to find the source code for this location. But it is clear that LdapExtLoginModule does set the context classloader to null in validatePassword. I haven't come up with any way of avoiding this.

While trying to circumvent this bug I tried to avoid following the AD referral. This doesn't seem to be possible, though. When setting "java.naming.referral" to "ignore", you would expect that the login would succeed. But as documented at http://docs.oracle.com/javase/jndi/tutorial/ldap/referral/jndi.html , some LDAP implementations might still throw a PartialResultException. This is indeed what I get:

{code}
Caused by: javax.naming.PartialResultException: Unprocessed Continuation Reference(s); remaining name 'dc=global,dc=scd,dc=company,dc=com'
{code} 

Spring points this out at http://static.springsource.org/spring-ldap/site/apidocs/org/springframework/ldap/core/LdapTemplate.html and has a way of supressing these exceptions:  "ignorePartialResultException".

With JBoss lacking this, I am stuck between a rock and a hard place. I cannot enable referrals due to the null context class loader bug, and I cannot disable them due to the PartialResultException bug.

So I would call this one a blocker. Any suggestions are greatly appreciated, as we are stuck upgrading to AS 7. This is a regression, by the way, since "follow" used to work on AS 5.1.0.GA which we are upgrading from. 

The only way of avoiding this problem that I've found is to narrow the tree which you search through in AD in such a way that you avoid hitting the referrals at all. There are a couple of related bugs and forum posts (see for instance https://issues.jboss.org/browse/AS7-2085), but I don't think any of them really nailed the problem down. It's pretty tricky since you don't even get a relevant stacktrace unless you enable "throwValidateError".

Thanks