Caching non-IO failures using the "cache-failures" flag doesn't work. Possible root of the issue is catch block in org.jboss.as.domain.management.security.LdapCacheService.BaseSearchCache.CacheEntry.getSearchResult(LdapConnectionHandler, K):

} catch (NamingException e) {
  if (cacheFailures) {
    failure = e;
  }
  throw e;
}

The problem is the IOException is thrown instead of NamingException in org.jboss.as.domain.management.security.LdapUserSearcherFactory.LdapUserSearcherImpl.search(LdapConnectionHandler, String):
throw MESSAGES.userNotFoundInDirectory(suppliedName);

and the message defined in org.jboss.as.domain.management.DomainManagementMessages:
@Message(id = 15231, value = "User '%s' not found in directory.")
IOException userNotFoundInDirectory(String username);

The stack trace looks like:

java.io.IOException: JBAS015231: User 'test' not found in directory.
	at org.jboss.as.domain.management.security.LdapUserSearcherFactory$LdapUserSearcherImpl.search(LdapUserSearcherFactory.java:123)
	at org.jboss.as.domain.management.security.LdapUserSearcherFactory$LdapUserSearcherImpl.search(LdapUserSearcherFactory.java:67)
	at org.jboss.as.domain.management.security.LdapCacheService.internalSearch(LdapCacheService.java:199)
	at org.jboss.as.domain.management.security.LdapCacheService.access$800(LdapCacheService.java:54)
	at org.jboss.as.domain.management.security.LdapCacheService$BaseSearchCache$CacheEntry.getSearchResult(LdapCacheService.java:375)
	at org.jboss.as.domain.management.security.LdapCacheService$ByAccessCache.search(LdapCacheService.java:526)
	at org.jboss.as.domain.management.security.UserLdapCallbackHandler$LdapCallbackHandler.handle(UserLdapCallbackHandler.java:201)
	at org.jboss.as.domain.management.security.SecurityRealmService$1.handle(SecurityRealmService.java:173)
	at org.jboss.as.domain.http.server.security.BasicAuthenticator.checkCredentials(BasicAuthenticator.java:144)
	at org.jboss.com.sun.net.httpserver.BasicAuthenticator.authenticate(BasicAuthenticator.java:77)
	at org.jboss.as.domain.http.server.security.BasicAuthenticator._authenticate(BasicAuthenticator.java:103)
	at org.jboss.as.domain.http.server.security.BasicAuthenticator.authenticate(BasicAuthenticator.java:80)
	at org.jboss.sun.net.httpserver.AuthFilter.doFilter(AuthFilter.java:64)
	at org.jboss.com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:81)
	at org.jboss.sun.net.httpserver.ServerImpl$Exchange$LinkHandler.handle(ServerImpl.java:710)
	at org.jboss.com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:78)
	at org.jboss.as.domain.http.server.RealmReadinessFilter.doFilter(RealmReadinessFilter.java:47)
	at org.jboss.as.domain.http.server.DmrFailureReadinessFilter.doFilter(DmrFailureReadinessFilter.java:45)
	at org.jboss.com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:81)
	at org.jboss.sun.net.httpserver.ServerImpl$Exchange.run(ServerImpl.java:682)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at org.jboss.threads.JBossThread.run(JBossThread.java:122)