Bug 901248 (JBPAPP6-1699)

project_key: JBPAPP6

Our failover tests for remote stateful EJBs start with a set of these exceptions:

{code}
06:59:44,609 ERROR [org.jboss.remoting.remote.connection] (Remoting "config-based-ejb-client-endpoint" read-1) JBREM000200: Remote connection failed: javax.security.sasl.SaslException: Authentication failed: all available authentication mechanisms failed
06:59:44,611 INFO  [org.jboss.ejb.client.remoting.RemotingConnectionClusterNodeManager] (ejb-client-cluster-node-connection-creation-2-thread-2) Could not create a connection for cluster node ClusterNode{clusterName='ejb', nodeName='perf18', clientMappings=[ClientMapping{sourceNetworkAddress=/0:0:0:0:0:0:0:0, sourceNetworkMaskBits=0, destinationAddress='10.16.90.54', destinationPort=4447}], resolvedDestination=[Destination address=10.16.90.54, destination port=4447]} in cluster ejb
java.lang.RuntimeException: javax.security.sasl.SaslException: Authentication failed: all available authentication mechanisms failed
	at org.jboss.ejb.client.remoting.IoFutureHelper.get(IoFutureHelper.java:91)
	at org.jboss.ejb.client.remoting.RemotingConnectionClusterNodeManager.getEJBReceiver(RemotingConnectionClusterNodeManager.java:89)
	at org.jboss.ejb.client.ClusterContext$EJBReceiverAssociationTask.call(ClusterContext.java:406)
	at org.jboss.ejb.client.ClusterContext$EJBReceiverAssociationTask.call(ClusterContext.java:380)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:662)
Caused by: javax.security.sasl.SaslException: Authentication failed: all available authentication mechanisms failed
	at org.jboss.remoting3.remote.ClientConnectionOpenListener$Capabilities.handleEvent(ClientConnectionOpenListener.java:382)
	at org.jboss.remoting3.remote.ClientConnectionOpenListener$Capabilities.handleEvent(ClientConnectionOpenListener.java:225)
	at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:72)
	at org.xnio.channels.TranslatingSuspendableChannel.handleReadable(TranslatingSuspendableChannel.java:189)
	at org.xnio.channels.TranslatingSuspendableChannel$1.handleEvent(TranslatingSuspendableChannel.java:103)
	at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:72)
	at org.xnio.nio.NioHandle.run(NioHandle.java:90)
	at org.xnio.nio.WorkerThread.run(WorkerThread.java:187)
	at ...asynchronous invocation...(Unknown Source)
	at org.jboss.remoting3.EndpointImpl.doConnect(EndpointImpl.java:270)
	at org.jboss.remoting3.EndpointImpl.connect(EndpointImpl.java:386)
	at org.jboss.ejb.client.remoting.NetworkUtil.connect(NetworkUtil.java:151)
	at org.jboss.ejb.client.remoting.NetworkUtil.connect(NetworkUtil.java:132)
	at org.jboss.ejb.client.remoting.RemotingConnectionClusterNodeManager.getEJBReceiver(RemotingConnectionClusterNodeManager.java:87)
	... 7 more
{code}

As seen in https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/EAP6/view/EAP6-Failover/job/eap-6x-failover-ejb-ejbremote-jvmkill-repl-sync/15/console-perf17/

This is unexpected, since we don't use any auth -- the remoting subsystem is configured like this:

{code}
<subsystem xmlns="urn:jboss:domain:remoting:1.1">
    <connector name="remoting-connector" socket-binding="remoting"/>
</subsystem>
{code}

As can be seen in https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/EAP6/view/EAP6-Failover/job/eap-6x-failover-ejb-ejbremote-jvmkill-repl-sync/15/artifact/report/config/jboss-perf18/standalone-ha.xml

The exceptions do no (perceived) harm, but they really shouldn't be there in the first place.

EJB client configration looks like:

{code}

remote.connectionprovider.create.options.org.xnio.Options.SSL_ENABLED=false

remote.connections=perf18,perf19,perf20,perf21

invocation.timeout=120000

remote.connection.perf18.host=perf18
remote.connection.perf18.port=4447
remote.connection.perf18.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS=false
remote.connection.perf18.channel.options.org.jboss.remoting3.RemotingOptions.MAX_OUTBOUND_MESSAGES=30000

remote.connection.perf19.host=perf19
remote.connection.perf19.port=4447
remote.connection.perf19.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS=false
remote.connection.perf19.channel.options.org.jboss.remoting3.RemotingOptions.MAX_OUTBOUND_MESSAGES=30000

remote.connection.perf20.host=perf20
remote.connection.perf20.port=4447
remote.connection.perf20.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS=false
remote.connection.perf20.channel.options.org.jboss.remoting3.RemotingOptions.MAX_OUTBOUND_MESSAGES=30000

remote.connection.perf21.host=perf21
remote.connection.perf21.port=4447
remote.connection.perf21.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS=false
remote.connection.perf21.channel.options.org.jboss.remoting3.RemotingOptions.MAX_OUTBOUND_MESSAGES=30000
{code}

As can be seen in https://svn.devel.redhat.com/repos/jboss-qa/load-testing/sf-components/project-clustering-ejb3/trunk/src/main/resources/jboss-ejb-client.properties

Note that I only provided links to one job in Jenkins, but this happens in a _lot_ of other jobs.