Bug 810155 - Default configuration may not be optimal for a 2-node cluster
Default configuration may not be optimal for a 2-node cluster
Status: VERIFIED
Product: JBoss Data Grid 6
Classification: JBoss
Component: Infinispan (Show other bugs)
6.0.0
Unspecified Unspecified
low Severity low
: ER1
: 6.0.1
Assigned To: Tristan Tarrant
nobody nobody
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-04-05 05:53 EDT by Michal Linhard
Modified: 2014-03-17 00:03 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
A sample configuration is now included in the JBoss Data Grid Server distribution called docs/examples/configs/standalone-two-nodes.xml, which provides better performance when using clusters formed by only two nodes. This configuration changes the default JGroups configuration to use the UNICAST protocol instead of the UNICAST2 and RSVP combination of protocols. For larger clusters the UNICAST2 and RSVP combination is recommended.
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
JGroups UNICAST UDP config snippet (1.37 KB, text/plain)
2012-06-08 09:10 EDT, Michal Linhard
no flags Details

  None (edit)
Description Michal Linhard 2012-04-05 05:53:08 EDT
In performance tables of issue:
https://bugzilla.redhat.com/show_bug.cgi?id=758759

we have worse results with ER6 config compared to ER4 config.
Comment 1 Michal Linhard 2012-04-05 05:54:24 EDT
I'll try to obtain improved JProfiler snapshots (based on sampling for this)
Comment 2 Michal Linhard 2012-04-05 06:03:11 EDT
Workaround: for size2 installations use JGroups config with UNICAST if it performs better.
Comment 3 Michal Linhard 2012-04-05 06:03:11 EDT
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Hot rod client stress tests of cluster of size 2 perform better with ER4 JGroups config (UNICAST) vs ER6 (UNICAST2+RSVP)
Comment 4 prabhat jha 2012-05-11 11:08:52 EDT
Changed target release to 6.0.0.GA.
Comment 5 prabhat jha 2012-05-29 13:13:43 EDT
Ondra, pls update the status once you run ER10 with ER4 jgroups config for 2 node cluster.
Comment 6 Ondrej Nevelik 2012-05-31 03:15:54 EDT
[1] is a throughput chart of a client stress test with Hotrod, dist, cluster size2 and jgroups subsystem config used in ER4. It can handle 25% more clients then when we use our common config although the max throughput is approximately the same.

[1] https://hudson.qa.jboss.com/hudson/view/EDG6/view/EDG-REPORTS-PERF/job/edg-60-perf-client-stress-test-hotrod/279/artifact/report/throughput.png
Comment 7 prabhat jha 2012-05-31 16:11:33 EDT
Ondra, we would need to run this again with ER11. If this is still the case then your observation needs to be documented in tuning guide.
Comment 8 Misha H. Ali 2012-06-03 20:58:28 EDT
    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1 +1 @@
-Hot rod client stress tests of cluster of size 2 perform better with ER4 JGroups config (UNICAST) vs ER6 (UNICAST2+RSVP)+Hot rod client stress tests with a two node cluster results in better performance with the ER4 JGroups configuration (using UNICAST) than the ER6 JGroups configuration (using UNICAST2 and RSVP).
Comment 9 Ondrej Nevelik 2012-06-06 04:13:31 EDT
For ER11 the conclusion is that for cluster size 2 the performance is better with jgroups configuration used with ER4 while it doesn't improve size 4 perf.
Comment 10 Misha H. Ali 2012-06-06 23:31:15 EDT
Prabhat's very valid point about external parties not knowing what we refer to by ERX:

"ER releases are internal and customers wont understand what we mean by these. I suggest this to convey "Out of box configuration is not optimal for a 2-node cluster." Ideally this should mention code snippet with right configuration or point to some URL which  has the configuration."

Flagging Michal as he provided the original information for this bug.
Comment 11 Michal Linhard 2012-06-08 09:10:51 EDT
Created attachment 590427 [details]
JGroups UNICAST UDP config snippet
Comment 12 Michal Linhard 2012-06-08 09:16:10 EDT
    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1 +1 @@
-Hot rod client stress tests with a two node cluster results in better performance with the ER4 JGroups configuration (using UNICAST) than the ER6 JGroups configuration (using UNICAST2 and RSVP).+Out-of-the-box JGroups configuration (using UNICAST2 and RSVP) might not lead to optimal performance in 2-node cluster scenario. We've found that 2-node clusters perform better with UNICAST and without RSVP. See attached JGroups subsystem configuration snippet "JGroups UNICAST UDP config snippet".
Comment 13 Misha H. Ali 2012-06-12 01:14:05 EDT
    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1 +1 @@
-Out-of-the-box JGroups configuration (using UNICAST2 and RSVP) might not lead to optimal performance in 2-node cluster scenario. We've found that 2-node clusters perform better with UNICAST and without RSVP. See attached JGroups subsystem configuration snippet "JGroups UNICAST UDP config snippet".+The default JBoss Data Grid JGroups configuration (using UNICAST2 and RSVP) does not lead to optimal performance in a two node cluster scenario. Two node clusters show improved performance with UNICAST and without RSVP.
Comment 15 Michal Linhard 2012-06-14 03:47:45 EDT
In hyperion size2 tests the performance difference isn't that significant
http://www.qa.jboss.com/~mlinhard/hyperion/run190-client-stress-02-CR2-unicast-no-rsvp/compare/merged-throughput.png

The two runs differ only in configuration of JGroups stack in the two protocols (UNICAST vs UNICAST2+RSVP), every other setting I've synced.
http://www.qa.jboss.com/~mlinhard/hyperion/run190-client-stress-02-CR2-unicast-no-rsvp/compare/configs/standalone_unicast.xml
http://www.qa.jboss.com/~mlinhard/hyperion/run190-client-stress-02-CR2-unicast-no-rsvp/compare/configs/standalone_unicast2+rsvp.xml

Perf difference (UNICAST vs UNICAST2+RSVP)
max number of clients: 26000 vs 23500 (10.6 %)
max throughput: 132609.20 vs 129351.05 (2.5 %)
Comment 16 Martin Gencur 2012-07-30 02:50:35 EDT
The JDG 6.0.1 ER1 server distribution now contains a sample configuration file ($JDG_HOME/docs/examples/configs/standalone-two-nodes.xml) with so far the most optimal configuration for 2-node cluster. I'm closing this for now as verified but in the future we should find out if there's not even a better configuration in terms of performance.

Note You need to log in before you can comment on or make changes to this bug.