Red Hat Bugzilla – Bug 361761
Support the active use of multiple network interfaces.
Last modified: 2016-04-26 09:24:57 EDT
One way to increase the reliability of cluster interconnect is to use multiple
physical connections. Currently network bonding can be used. An alternative is
to support the knowledge of the mulitple interfaces in the cluster
infrastructure layers of openais and cman.
This may well work at the moment but:
1) Must use SCTP (which had serious problems in 5.0). TCP is currently the
default, though sctp is an unsupported option
2) Conga does not support it's configuration
3) There has been no QE testing
4) There is no documentation.
We should validate that this works, add Conga support, test the solution, and
document it's use.
It seems to me that dlm_controld, at least, needs some work here as it only
seems to support a single IP address per node. I also reckon that it should
automatically switch over to sctp if more than one address is passed back from cman.
I did a basic test of single-home over sctp and encountered some troubles! I
know this used to work so more investigation is needed.
OK it looks like my problems were down to insufficient receive memory in the
sockets. dlm_controld (or maybe the startup script?) needs to do something like
the following to make sctp work correctly.
echo 4194304 > /proc/sys/net/core/rmem_default
echo 4194304 > /proc/sys/net/core/rmem_max
Added multi-home code to dlm_controld on CVS head:
Checking in dlm_daemon.h;
/cvs/cluster/cluster/group/dlm_controld/dlm_daemon.h,v <-- dlm_daemon.h
new revision: 1.14; previous revision: 1.13
Checking in member_cman.c;
/cvs/cluster/cluster/group/dlm_controld/member_cman.c,v <-- member_cman.c
new revision: 1.9; previous revision: 1.8
Added the networking params to the init script. NOTE: someone who actually knows
about init scripts might like to check this!
On CVS head:
Checking in cman;
/cvs/cluster/cluster/cman/init.d/cman,v <-- cman
new revision: 1.33; previous revision: 1.32
Basic tests of multi-run openais seem to work, but see bz#424681: the default
build restricts you to 2 rings.
Also using "rrp_mode: active" doesn't work if the multicast and port numbers are
the same for all rings. Oddly (to me), "rrp_mode: passive" does work when
configured like this. Changing the multicast address and/or port numbers for
each ring works fine.
With more than one interface configured cman switches to "rrp_mode: active" so
we either need to document this, fix it (and make conga check it) or make cman
Opened for public viewing.
Assigning to sdake (sorry steve) so he can check the multi-ring.
I will test it. One of my switches failed a few months ago so I haven't had a
mulitring setup available to set at all. So I'll have to work on getting a new
Are you sure your not having both NICs connected to the same interconnect? That
would explain the port/ring behavior you have in active mode.
In order for redundant ring to work, the interconnects must be completely
seperated, or they must use different port or multicast addresses.
Closing this as not a bug for the cman component. It is believed this is just a
configuration/setup/management issue and the cluster project wiki has a
description of how to perform this work. So the work is in the user interfaces
for conga/system-config-cluster and there is already a defect recorded for those