Bug 2037904 - upgrade operator deployment failed due to memory limit too low for manager container
Summary: upgrade operator deployment failed due to memory limit too low for manager co...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Telco Edge
Version: 4.10
Hardware: Unspecified
OS: Unspecified
urgent
high
Target Milestone: ---
: 4.10.0
Assignee: Ian Miller
QA Contact: yliu1
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-01-06 19:36 UTC by yliu1
Modified: 2022-03-10 16:37 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-03-10 16:37:33 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift-kni cluster-group-upgrades-operator pull 71 0 None Merged Bug 2037904: Remove pod limits from operator 2022-01-26 21:59:52 UTC
Red Hat Product Errata RHSA-2022:0056 0 None None None 2022-03-10 16:37:45 UTC

Description yliu1 2022-01-06 19:36:58 UTC
Description of problem:
Upgrade operator pod is in crashloopback after deployed with OOMKilled error. 

The memory limit defined in manager container is too low (30M):
https://github.com/openshift-kni/cluster-group-upgrades-operator/blob/main/config/manager/manager.yaml#L51


Version-Release number of selected component (if applicable):
4.10

How reproducible:
100% in my env

Steps to Reproduce:
1.clone upgrade operator repo, and follow instructions to build and deploy it: https://github.com/openshift-kni/cluster-group-upgrades-operator#how-to-deploy
2. check deployed resources

Actual results:
pod in crashloopback due to OOMKilled

        terminated:
          containerID: cri-o://83ef8df7d68111459978511ce7f2b02d669ec50b5baff98ef6435655700cf77e
          exitCode: 137
          finishedAt: "2022-01-06T19:08:04Z"
          reason: OOMKilled
          startedAt: "2022-01-06T19:07:40Z"
      name: manager
      ready: false
      restartCount: 9
      started: false


Expected results:
deployment succeeded

Additional info:
workaround: 
Bump up memory limit via "oc edit deployments.apps -n openshift-cluster-group-upgrades"
Then delete old pods and replicas.

Comment 3 yliu1 2022-02-04 17:44:52 UTC
Verified with latest TALO. 
TALO pod now has only resource request without limit. 

    resources:
      requests:
        cpu: 100m
        memory: 20Mi

Comment 5 errata-xmlrpc 2022-03-10 16:37:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0056


Note You need to log in before you can comment on or make changes to this bug.