Red Hat Bugzilla – Bug 1475034
Metrics chart reporting 74000 Millicores for an app running on a node with only 8 cores
Last modified: 2017-11-03 09:43:28 EDT
Description of problem:
application with several replications running just fine suddenly has metrics reporting significantly more cores that is possible (node has 8 cores, app reported 74,000 millicores).
Version-Release number of selected component (if applicable):
OpenShift Container Platform 18.104.22.168
Attaching files shortly
@sross: it looks like Heapster is using 15s for its interval, and I believe at this interval we can sometimes get strange cpu usage results back. Is this something we have seen before? A very large cpu spike which is nonsense.
those logs do not look like a healthy Heapster :-/
I'd try switching to an interval of 30s, as well as checking what the summary endpoint says, and what happens if you switch to using the summary source (`--source=kubernetes.summary_api:...` instead of `--source=kubernetes:...`.
We've seen spikes like that due to bad (non-monotonically increasing) CPU metrics and overflow, or occasionally due to bad metrics coming from Kubelet/cAdvisor, but I thought we'd fixed most of those issues.