Bug 745813 (EDG-71)

Summary: Provide default JGroups stack config which can be used with ServiceBindingManager
Product: [JBoss] JBoss Data Grid 5 Reporter: Richard Achmatowicz <rachmato>
Component: InfinispanAssignee: Default User <jbpapp-maint>
Status: CLOSED NEXTRELEASE QA Contact:
Severity: high Docs Contact:
Priority: high    
Version: unspecified   
Target Milestone: ---   
Target Release: EAP 5.1.0 EDG TP   
Hardware: Unspecified   
OS: Unspecified   
URL: http://jira.jboss.org/jira/browse/EDG-71
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-10-26 17:01:36 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Richard Achmatowicz 2010-10-20 20:37:33 UTC
project_key: EDG

In the Infinispan configuration file, we may specify a JGroups transport by coding:

<global>
 ...
 <transport clusterName="infinispan-cluster" distributedSyncTimeout="50000" nodeName="Jalapeno">
   <properties>
      <property name="configurationFile" value="udp.xml"/>
   </properties>
 </transport>
 ...
</global>

where the missing <transport/> element attribute called  transportClass defaults to  the correct value for using a JGroups transport.  If we in turn leave out the configuration file specification, the  configuration file used defaults to jgroups-udp.xml.

The jgroups-udp.xml file is not suitable for use with EAP 5 and ServiceBindingManager, as it includes the empty protocol:layer <FD_SOCK/>. FD_SOCK opens a port and if two server instances are bound to the same IP address using SBM, we will have port conflits. 

The file should be modified to use the SBM system property:

<FD_SOCK start_port="${jboss.jgroups.udp.fd_sock_port:54200}"/>

The same could be said for:

- diagnostics when it is enabled
     enable_diagnostics="${jboss.jgroups.enable_diagnostics:true}"
     diagnostics_addr="${jboss.jgroups.diagnostics_addr:224.0.75.75}"
     diagnostics_port="${jboss.jgroups.diagnostics_port:7500}"

- the UDP unicast bind port
      bind_port="${jboss.jgroups.udp.bind_port:55200}"

Comment 1 Brian Stansberry 2010-10-20 21:15:19 UTC
That shouldn't cause port conflicts, since JGroups uses an ephemeral port if no bind port is specified. (Same is true in the transport protocols.) TCP requires a port config, but will scan for an available one if the configured one is not available.

This JIRA is still valid though, as using ephemeral ports is contrary to the way other EAP services are configured, including the JGroups stacks used in "all" and "production".

Comment 2 Galder Zamarreño 2010-10-25 14:00:28 UTC
Link: Added: This issue depends ISPN-733


Comment 3 Richard Achmatowicz 2010-10-25 20:19:03 UTC
Link: Added: This issue is related to JBPAPP-5255


Comment 4 Richard Achmatowicz 2010-10-25 20:34:39 UTC
Galder, while you are on this one, could you please check that the JGroups configuration file can be read from the infinispan-configuration.xml file.  I am persistently getting this error when I try to specify a JGroups config file using, say:

<transport clusterName="infinispan-cluster" distributedSyncTimeout="50000" nodeName="Jalapeno">
   <properties>
        <property name="configurationFile" value="jgroups-udp.xml"/>
   </properties>
</transport>

If I don't specify a JGroups configuration file, Infinispan starts OK.


from server/log/output.log
----------------------------
16:23:02,845 INFO  [LogNotificationListener] Adding notification listener for logging mbean "jboss.system:service=Logging,type=Log4jService" to server org.j\
boss.mx.server.MBeanServerImpl@18b3e62[ defaultDomain='jboss' ]
16:23:10,838 INFO  [ComponentsJmxRegistration] Could not register object with name: infinispan:cache-name=[global],jmx-resource=CacheManager
16:23:11,024 WARN  [GenericTransactionManagerLookup] Falling back to DummyTransactionManager from Infinispan
16:23:11,445 INFO  [JGroupsTransport] Starting JGroups Channel
16:23:11,465 ERROR [JGroupsTransport] Error while trying to create a channel using config files: jgroups-udp.xml
16:23:11,466 ERROR [AbstractKernelController] Error installing to Installed: name=HotRodServer state=Start
org.infinispan.CacheException: Unable to invoke method public abstract void org.infinispan.remoting.transport.Transport.start() on object
        at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:173)
        at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:852)
        at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:672)
        at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:574)
        at org.infinispan.factories.GlobalComponentRegistry.start(GlobalComponentRegistry.java:134)
        at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:135)
        at org.infinispan.CacheDelegate.start(CacheDelegate.java:305)
        at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:449)
        at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:412)
        at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:389)
        at org.infinispan.server.core.AbstractProtocolServer.startDefaultCache(AbstractProtocolServer.scala:97)
        at org.infinispan.server.core.AbstractProtocolServer.start(AbstractProtocolServer.scala:64)
        at org.infinispan.server.hotrod.HotRodServer.start(HotRodServer.scala:42)
        at org.infinispan.server.core.AbstractProtocolServer.start(AbstractProtocolServer.scala:83)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)






Comment 5 Galder Zamarreño 2010-10-26 15:54:28 UTC
Just tried that and the actual exception is:

Caused by: java.lang.NullPointerException: the specifed protocol stack configuration was null
	at org.jgroups.conf.ConfiguratorFactory.checkForNullConfiguration(ConfiguratorFactory.java:381)
	at org.jgroups.conf.ConfiguratorFactory.getStackConfigurator(ConfiguratorFactory.java:72)
	at org.jgroups.JChannel.<init>(JChannel.java:212)
	at org.infinispan.remoting.transport.jgroups.JGroupsTransport.buildChannel(JGroupsTransport.java:252)

This is a bit odd. Basically, it's saying the file is not there. However, if u set up with this, then it works: 

        <property name="configurationFile" value="config-samples/jgroups-udp.xml"/> 

This file is located within the infinispan-core.jar, maybe it has changed locations in recent versions.

Anyway, for this JIRA we'll be using diff files so nothing to worry at least for this jira.

Comment 6 Galder Zamarreño 2010-10-26 16:09:00 UTC
Link: Added: This issue related ISPN-736


Comment 7 Richard Achmatowicz 2010-10-26 20:23:12 UTC
So does that mean that if I (an EDG user) want to override the jgroups configuration file, I have to place it in the Infinispan jar in the config-samples directory? That's rather inconvenient.

Galder, could you please clarify the possible locations where these configuration files can be stored? (even if there is only one :-()





Comment 8 Galder Zamarreño 2010-10-27 09:25:57 UTC
Not at all. As per the email thread, I've ended up copy/pasting the stacks and putting them at the root of the sar so that they can be directly accessed from config. Afterwards, you can either change the files at the root of sar, or point the infinispan config to a different file, or pass -Djboss.infinispan.jgroups.config with the location. See the commit to find out more.

Comment 9 Richard Achmatowicz 2010-11-24 17:57:00 UTC
Link: Added: This issue is related to JBPAPP-5466


Comment 10 Richard Achmatowicz 2010-11-24 18:07:22 UTC
Link: Removed: This issue is related to JBPAPP-5466