Bug 1712293

Summary: "Memory Usage" under "Workloads -> Pods" of cluster console has large difference with "oc adm top pod"
Product: OpenShift Container Platform Reporter: Junqi Zhao <juzhao>
Component: MonitoringAssignee: Sergiusz Urbaniak <surbania>
Status: CLOSED NOTABUG QA Contact: Junqi Zhao <juzhao>
Severity: low Docs Contact:
Priority: low    
Version: 4.1.0CC: anpicker, erooth, mloibl, pkrupa, surbania
Target Milestone: ---   
Target Release: 4.5.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-05-05 07:49:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
console screenshot none

Description Junqi Zhao 2019-05-21 09:27:30 UTC
Description of problem:
Login cluster console with cluster admin, click "Workloads -> Pods", and select one project from drop-down list, eg: openshift-kube-apiserver,
then select one pod, in the "Pod Overview" section, the "Memory Usage" uses 
"pod_name:container_memory_usage_bytes:sum{pod_name='kube-apiserver-ip-10-0-139-94.us-east-2.compute.internal',namespace='openshift-kube-apiserver'}" to calculate "Memory Usage"

search in prometheus UI
****************************************************************
pod_name:container_memory_usage_bytes:sum{pod_name='kube-apiserver-ip-10-0-139-94.us-east-2.compute.internal',namespace='openshift-kube-apiserver'} / 1024 /1024

Element	Value
{namespace="openshift-kube-apiserver",pod_name="kube-apiserver-ip-10-0-139-94.us-east-2.compute.internal"}	1833.5234375
****************************************************************

whereas
****************************************************************
# oc -n openshift-kube-apiserver adm top pod kube-apiserver-ip-10-0-139-94.us-east-2.compute.internal
NAME                                                       CPU(cores)   MEMORY(bytes)   
kube-apiserver-ip-10-0-139-94.us-east-2.compute.internal   206m         867Mi  
****************************************************************

Although we know `oc -n openshift-kube-apiserver adm top pod kube-apiserver-ip-10-0-139-94.us-east-2.compute.internal` uses the following expression
****************************************************************
sum(container_memory_working_set_bytes{container_name!="POD",container_name!="",pod_name!="", pod_name="kube-apiserver-ip-10-0-139-94.us-east-2.compute.internal"}) / 1024 /1024
Element	Value
{}	867.28515625
****************************************************************

I think "Memory Usage" under "Workloads -> Pods" should be the same with "oc adm top pod"

Version-Release number of selected component (if applicable):
4.1.0-0.nightly-2019-05-18-050636

How reproducible:
Always

Steps to Reproduce:
1. See the description part
2.
3.

Actual results:
"Memory Usage" under "Workloads -> Pods" is not the same with "oc adm top pod"

Expected results:
"Memory Usage" under "Workloads -> Pods" should be the same with "oc adm top pod"

Additional info:

Comment 2 Pawel Krupa 2019-05-21 12:20:41 UTC
Similar issue with CPU metrics: https://bugzilla.redhat.com/show_bug.cgi?id=1709176

Comment 9 Sergiusz Urbaniak 2019-09-04 06:54:32 UTC
Created attachment 1611369 [details]
console screenshot

Comment 12 Sergiusz Urbaniak 2019-09-04 11:15:19 UTC
*** Bug 1709176 has been marked as a duplicate of this bug. ***

Comment 13 Junqi Zhao 2019-09-05 11:25:25 UTC
fix is not in 4.2.0-0.nightly-2019-09-04-142146