Bug 1895532 - HPA monitoring cpu utilization fails for deployments which have init containers
Summary: HPA monitoring cpu utilization fails for deployments which have init containers
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 4.5
Hardware: x86_64
OS: Linux
Target Milestone: ---
: 4.6.z
Assignee: Joel Smith
QA Contact: Weinan Liu
Depends On: 1867477
Blocks: 1897313
TreeView+ depends on / blocked
Reported: 2020-11-06 23:30 UTC by OpenShift BugZilla Robot
Modified: 2024-06-13 23:22 UTC (History)
24 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2020-11-30 16:45:31 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Github openshift k8s-prometheus-adapter pull 36 0 None closed [release-4.6] Bug 1895532: Populate both CPU and Memory resource container metrics if one is specified 2021-02-19 17:14:47 UTC
Red Hat Knowledge Base (Solution) 5972361 0 None None None 2021-04-16 14:30:35 UTC
Red Hat Product Errata RHBA-2020:5115 0 None None None 2020-11-30 16:45:51 UTC

Comment 5 Weinan Liu 2020-11-18 14:49:21 UTC
All look good execept for the warning (the same with on 4.7)

  Type     Reason                        Age                 From                       Message
  ----     ------                        ----                ----                       -------
  Warning  FailedComputeMetricsReplicas  18m (x12 over 20m)  horizontal-pod-autoscaler  invalid metrics (1 invalid out of 1), first error is: failed to get cpu utilization: did not receive metrics for any ready pods
  Warning  FailedGetResourceMetric       17m (x13 over 20m)  horizontal-pod-autoscaler  did not receive metrics for any ready pods

Since we provide              "cpu": "0",

While do we still have the warning?

Comment 6 Joel Smith 2020-11-18 15:11:04 UTC
The warning is harmless as long as the age is about the same as the age of the pods.  What happens is that if the pods are created after the HPA or at about the same time, the HPA emits the warning for the first few minutes until the pod metrics are available.  There is always a delay after creating a pod before its metrics become available.

If you create the pods, then wait 5 minutes, then create the HPA, I think you won't see these warnings. That's because the HPA will be able to get metrics from the very beginning.

Comment 7 Weinan Liu 2020-11-18 16:27:16 UTC
Verifed as per comment #5 and #6

Comment 9 errata-xmlrpc 2020-11-30 16:45:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6.6 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.