In haproxy, a single proxy has a maximum number of concurrent connections. By default this is 2000. This can be set using the 'maxconn' parameter in a proxy's listen (or frontend) block in haproxy.cfg. [Note: this is not to be confused with haproxy's global maxconn parameter] Typically the default of 2000 concurrent connections is not an issue, but for large scale deployments where 1) hardware resources (CPU cores, memory, etc) allow for many processes for OpenStack services and 2) mariadb is set to have high number of max connections, the haproxy limit will result in connection refused by haproxy. In short, haproxy should not be the limiting factor for database connections. The 'maxconn' parameter should be set sufficiently high so that haproxy never causes the connection to be refused, but rather the database service should enfore this. To do this, the galera proxy should have maxconn set to at least what max_connections is set to in /etc/my.cnf.d/server.cnf.
(In reply to Ryan O'Hara from comment #0) > In haproxy, a single proxy has a maximum number of concurrent connections. > By default this is 2000. This can be set using the 'maxconn' parameter in a > proxy's listen (or frontend) block in haproxy.cfg. [Note: this is not to be > confused with haproxy's global maxconn parameter] > > Typically the default of 2000 concurrent connections is not an issue, but > for large scale deployments where 1) hardware resources (CPU cores, memory, > etc) allow for many processes for OpenStack services and 2) mariadb is set > to have high number of max connections, the haproxy limit will result in > connection refused by haproxy. In short, haproxy should not be the limiting > factor for database connections. The 'maxconn' parameter should be set > sufficiently high so that haproxy never causes the connection to be refused, > but rather the database service should enfore this. To do this, the galera > proxy should have maxconn set to at least what max_connections is set to in > /etc/my.cnf.d/server.cnf. We can do this, repasting my comment from a related BZ [1] to make sure you see it and comment if needed: [1] https://bugzilla.redhat.com/show_bug.cgi?id=1240826#c11 " The only connection limit we set is for the global maxconn setting, which is 10000 in ofi[1]. This issue looks to be tracked in a new bug from Ryan [2], so apparently this setting is not enough. If what Giulio did for tripleo [3] is reasonable, we can do a variation of that for ofi. Note that Ryan's BZ is against OSP 6, so I am not sure if we might need to clone it to track it into OFI's OSP 7 release separately. If Giulio's settings are what we want, that is a relatively straight forward addition here as well. Since our puppet is structured a bit differently that tripleo's, we could also do this by setting the haproxy connection limit for galera to whatever was passed in for max_connections (or some calculation based on that number), preventing an additional parameter being needed for the user to set. [1] https://github.com/redhat-openstack/astapor/blob/master/puppet/modules/quickstack/manifests/load_balancer/common.pp#L9 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1244929 [3] https://review.openstack.org/#/c/202525/1 "
Patch posted: https://github.com/redhat-openstack/astapor/pull/552
Merged