Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1850270

Summary: UI reports different memory and cpu values compared to command line output
Product: OpenShift Container Platform Reporter: Sam Yangsao <syangsao>
Component: Management ConsoleAssignee: Andrew Pickering <anpicker>
Status: CLOSED DUPLICATE QA Contact: Yadan Pei <yapei>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.4CC: anpicker, aos-bugs, jokerman, pkrupa, pweil, spadgett
Target Milestone: ---Keywords: Reopened
Target Release: 4.6.0   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-06-26 12:55:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
output of UI and command line output none

Description Sam Yangsao 2020-06-23 20:45:02 UTC
Created attachment 1698499 [details]
output of UI and command line output

Description of problem:

UI reports different memory and cpu values compared to command line output

Version-Release number of selected component (if applicable):

OCP 4.4.9

How reproducible:

Always

Steps to Reproduce:

1.  Install Elasticsearch following [1] with storage set to `EmptyDir`
2.
3.

[1] https://docs.openshift.com/container-platform/4.4/logging/config/cluster-logging-elasticsearch.html#cluster-logging-elasticsearch-persistent-storage-empty_cluster-logging-elasticsearch

Actual results:

UI reports the Elasticsearch pod is using:

8945.2 MiB
1.940 Cores

While in command line the node reports using:

[...]
Non-terminated Pods:                      (11 in total)
  Namespace                               Name                                             CPU Requests  CPU Limits  Memory Requests  Memory Limits  AGE
  ---------                               ----                                             ------------  ----------  ---------------  -------------  ---
  openshift-cluster-node-tuning-operator  tuned-9czzr                                      10m (0%)      0 (0%)      50Mi (0%)        0 (0%)         4d5h
  openshift-dns                           dns-default-84gcw                                110m (3%)     0 (0%)      70Mi (0%)        512Mi (2%)     4d5h
  openshift-image-registry                node-ca-djc4b                                    10m (0%)      0 (0%)      10Mi (0%)        0 (0%)         4d5h
  openshift-logging                       elasticsearch-cdm-1kapyvbj-3-7946bd6446-dsnp6    1100m (31%)   0 (0%)      16448Mi (71%)    16448Mi (71%)  14m **** HERE ****
  openshift-logging                       fluentd-nph4p                                    100m (2%)     0 (0%)      736Mi (3%)       736Mi (3%)     14m
  openshift-logging                       kibana-674d99cc9f-kv46t                          200m (5%)     0 (0%)      992Mi (4%)       992Mi (4%)     15m
[...]

71% is about 11678 MiB
CPU Requests show 341m

Expected results:

Customer expects utilization numbers to match up either via the UI and command line

Additional info:

Picture of outputs above are attached

Comment 2 Samuel Padgett 2020-06-24 14:02:09 UTC
Hi, requests and limits are different from actual usage. These values are not expected to be the same.

Requests and limits tell k8s how much resources your pod needs so it can be placed on a node with enough resources available. These are set in the workload YAML and don't change. See https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/

The console is showing you actual usage for your running pod, which will be different from requests.

Comment 3 Sam Yangsao 2020-06-24 14:31:11 UTC
(In reply to Samuel Padgett from comment #2)
> Hi, requests and limits are different from actual usage. These values are
> not expected to be the same.
> 

Understood.

> Requests and limits tell k8s how much resources your pod needs so it can be
> placed on a node with enough resources available. These are set in the
> workload YAML and don't change. See
> https://kubernetes.io/docs/concepts/configuration/manage-resources-
> containers/
> 
> The console is showing you actual usage for your running pod, which will be
> different from requests.
 
The confusing part here is we're seeing the percentage values from the `oc describe node ...` output.

What do these correlate to?

<snip>

  Namespace                               Name                                             CPU Requests  CPU Limits  Memory Requests  Memory Limits  AGE
  ---------                               ----                                             ------------  ----------  ---------------  -------------  ---
[...]
  openshift-logging                       elasticsearch-cdm-1kapyvbj-3-7946bd6446-dsnp6    1100m (31%)   0 (0%)      16448Mi (71%)    16448Mi (71%)  14m **** HERE ****

</snip>

So 31% of 1100m is 341m, correct?  Or should we be ignoring this value since it doesn't match up to actual usage in the UI.

Thanks!

Comment 4 Samuel Padgett 2020-06-24 14:53:16 UTC
Sorry, I missed that you were referring to the percentages. What CLI command are you running?

Andy, are you able to help us look at this?

Comment 5 Samuel Padgett 2020-06-24 14:54:12 UTC
OK, I see it's `oc describe node`.

Comment 6 Sam Yangsao 2020-06-24 14:57:28 UTC
I'm also wondering, is there another command that we can run to see the same values represented in the UI?

Thanks!

Comment 7 Samuel Padgett 2020-06-24 15:15:46 UTC
Here are the Prometheus queries we're running:

https://github.com/openshift/console/blob/master/frontend/public/components/pod.tsx#L62-L75

Comment 8 Sam Yangsao 2020-06-24 15:33:24 UTC
Setting `needinfo` for comment#4

Comment 9 Pawel Krupa 2020-06-26 09:54:55 UTC
In 4.5 CPU usage queries which are behind `oc top` (also used in `oc describe`) changed after https://bugzilla.redhat.com/show_bug.cgi?id=1812004. They were changed to https://github.com/openshift/cluster-monitoring-operator/blob/release-4.5/assets/prometheus-adapter/config-map.yaml#L7-L8

Comment 10 Samuel Padgett 2020-06-26 12:55:41 UTC

*** This bug has been marked as a duplicate of bug 1812004 ***