Bug 2017708
Summary: | Too many haproxy processes in default-router pod causing high load average after upgrade from v4.8.3 to v4.8.10 | ||
---|---|---|---|
Product: | OpenShift Container Platform | Reporter: | Andrew McDermott <amcdermo> |
Component: | Networking | Assignee: | Andrew McDermott <amcdermo> |
Networking sub component: | router | QA Contact: | Arvind iyengar <aiyengar> |
Status: | CLOSED ERRATA | Docs Contact: | |
Severity: | high | ||
Priority: | high | CC: | aiyengar, aos-bugs, hongli |
Version: | 4.8 | ||
Target Milestone: | --- | ||
Target Release: | 4.8.z | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | Release Note | |
Doc Text: |
Prior to 4.8 the default balancing algorithm was "leastconn". The
default was changed to "random" in OpenShift 4.8.0 for non-passthrough
routes.
Switching to "random" significantly increases memory consumption for
each and every HAProxy process and the cumulative memory usage can be
significant particularly if you have a lot of websocket connections.
To mitigate this significant memory consumption the default balancing
algorithm has now been reverted to "leastconn" for OpenShift 4.8 and
4.9.
Once we have a solution that does not incur such significant memory
requirements we would default to "random" again, but that would be for
a future OpenShift release (i.e., 4.10 or later).
You can check the default setting via:
$ oc get deployment -n openshift-ingress router-default -o yaml | grep -A 2 ROUTER_LOAD_BALANCE_ALGORITHM
- name: ROUTER_LOAD_BALANCE_ALGORITHM
value: leastconn
The "random" option is still available but each and every route that
would benefit from this algorithmic choice will now have to explicitly
request that by setting the following annotation on a per-route basis:
$ oc annotate -n <NAMESPACE> route/<ROUTE-NAME> "haproxy.router.openshift.io/balance=random"
|
Story Points: | --- |
Clone Of: | Environment: | ||
Last Closed: | 2021-11-11 20:12:09 UTC | Type: | --- |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | 2015829 | ||
Bug Blocks: |
Comment 3
Hongan Li
2021-10-29 08:55:16 UTC
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 4.8.19 bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:4109 |