Bug 2032006 - openshift-gitops-application-controller-0 failed to schedule with sufficient node allocatable resource
Summary: openshift-gitops-application-controller-0 failed to schedule with sufficient ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Telco Edge
Version: 4.10
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 4.10.0
Assignee: melserng
QA Contact: yliu1
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-12-13 20:46 UTC by yliu1
Modified: 2022-03-12 04:39 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-03-12 04:39:26 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift-kni cnf-features-deploy pull 867 0 None Merged Bug: 2032006 ztp: set ArgoCD resources request 2022-01-07 13:44:15 UTC
Red Hat Product Errata RHSA-2022:0056 0 None None None 2022-03-12 04:39:46 UTC

Description yliu1 2021-12-13 20:46:39 UTC
Description of problem:

openshift-gitops-application-controller-0 stuck at pending with insufficient memory warning. However, the allocatable memory for hub cluster node indicates more than requested memories. 


Version-Release number of selected component (if applicable):
hub version: 4.9.8
ACM version: 2.4.1
gitops version: 1.3.1
cnf-feature-deploy: master (4.10)


How reproducible: 100%


Steps to Reproduce:
1. trigger sno install using ztp image built from latest cnf-feature-deploy contents
2. check gitops apps and pods

Actual results:
2. openshift-gitops-application-controller-0 stuck at pending

// pod resource requests:
    Limits:
      cpu:     16
      memory:  16Gi
    Requests:
      cpu:        8
      memory:     8Gi
// Pod events
Events:
  Type     Reason            Age    From               Message
  ----     ------            ----   ----               -------
  Warning  FailedScheduling  9m59s  default-scheduler  0/3 nodes are available: 1 Insufficient cpu, 3 Insufficient memory.
  Warning  FailedScheduling  8m57s  default-scheduler  0/3 nodes are available: 1 Insufficient cpu, 3 Insufficient memory.

// node allocatable resources:
  allocatable:
    cpu: 15500m
    ephemeral-storage: "76754504782"
    hugepages-1Gi: "0"
    hugepages-2Mi: "0"
    memory: 19364776Ki
    pods: "250"
  capacity:
    cpu: "16"
    ephemeral-storage: 81332Mi
    hugepages-1Gi: "0"
    hugepages-2Mi: "0"
    memory: 20515752Ki
    pods: "250"


Expected results:
openshift-gitops-application-controller-0 should be successfully scheduled.

Additional info:
This issue started to show up after resource requests and limits were increased for this pod. 
It used request 1Gi memory with 2Gi limits, which worked fine.

Comment 2 yliu1 2022-02-01 17:57:48 UTC
Verified on ocp 4.10.0-fc.2 with ztp container image built from master branch.

Comment 5 errata-xmlrpc 2022-03-12 04:39:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0056


Note You need to log in before you can comment on or make changes to this bug.