Bug 1401010 - Memory and CPU donut shows nonsense value
Summary: Memory and CPU donut shows nonsense value
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: Frontend.Core
Version: 4.0.5
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: ovirt-4.1.1
: 4.1.1
Assignee: Shirly Radco
QA Contact: Lucie Leistnerova
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-12-02 14:52 UTC by Michal Skrivanek
Modified: 2017-04-21 09:42 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: The dashboard host utilization queries were calculated as the sum of percentages of memory/cpu usage per each host, of the total sum of memory/cpu of those hosts. Consequence: The dashboard gauges are not calculated correctly Fix: Updated the queries to calculate the sum of used cpu/memory out of the total , to represent correctly the used cpu/memory. Result:
Clone Of:
Environment:
Last Closed: 2017-04-21 09:42:15 UTC
oVirt Team: UX
Embargoed:
rule-engine: ovirt-4.1+
rule-engine: blocker+
rule-engine: planning_ack+
sradco: devel_ack+
lsvaty: testing_ack+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 70703 0 master MERGED dashboards: Fixed host dashboard queries 2017-02-13 13:10:48 UTC
oVirt gerrit 72261 0 ovirt-engine-4.1 MERGED dashboards: Fixed host dashboard queries 2017-02-15 22:21:48 UTC

Description Michal Skrivanek 2016-12-02 14:52:59 UTC
Version: 4.0.5

The utilization seem to be calculated as a sum of percentages of memory usage per each host, of the total sum of memory of those hosts.

1st host 2GB RAM, 1 VM with 1GB RAM running, memory utilization is roughly 50%
2nd host 100GB RAM, 1 VM with 1 GB RAM running, memory utilization ~1%

donut will show ~52GB memory is utilized(51% of 102GB), and there is about 50GB available which is completely misleading

Comment 1 Oved Ourfali 2016-12-06 13:12:21 UTC
Shirly - can you look at the calculation and share your thoughts on what the query should be?

Comment 2 Filip Krepinsky 2017-01-06 15:42:43 UTC
Also I think the cpu values are wrong in the same way.

It is calculated as average of cpu percentages. I think there should be distinction between cpus with different number of cores at least.

So that means these should be accounted for:

cpu with 50% usage and 8 cores
cpu with 50% usage and 4 cores 


Also, can vm run on one thread?


I am making moVirt dashboard (REST API), so I will use adjusted percentages to sockets and cores.

> Shirly - can you look at the calculation and share your thoughts on what the query should be?

And "used memory" (in Bytes) of each host.


Shirly, please tell me if you come up with different calculations so that we can adjust both implementations.

Comment 3 Filip Krepinsky 2017-01-06 15:51:21 UTC
Also, should we count total cpu/memory (used in over commit) from all hosts? 
I think it makes sense to show only over commit for running hosts.

Comment 4 Filip Krepinsky 2017-01-06 17:17:38 UTC
I am also getting different values for number of cores from this query

HostDwhDAO.properties:

host.total_cpu_memory_count=SELECT \
            SUM(COALESCE(hosts.number_of_cores, 0) * COALESCE(hosts.number_of_sockets, 0)) AS cpu_total ...


When I call the REST API and make the calculations, I will get 62 cores on my setup. But this query returns 182 cores. Do you know what makes the difference?

Comment 5 Filip Krepinsky 2017-01-06 21:54:29 UTC
This is my solution how to get cpu usage from REST API:
https://github.com/suomiy/moVirt/blob/dashboardFix/moVirt/src/main/java/org/ovirt/mobile/movirt/ui/dashboard/general/DashboardGeneralFragment.java#L66-L77

*Note: entityCores == sockets * coresPerSocket

Comment 6 Shirly Radco 2017-01-08 08:01:33 UTC
You are currect. Already working on fixing these queries.

Comment 7 Filip Krepinsky 2017-01-09 13:06:08 UTC
Does the fixing include even the core count, or what is your opinion on this?

> When I call the REST API and make the calculations, I will get 62 cores on my setup. But this query returns 182 cores. Do you know what makes the difference?

Comment 8 Filip Krepinsky 2017-01-09 13:31:00 UTC
Are you going to also make changes to this?

> Also, should we count total cpu/memory (used in over commit) from all hosts?

Comment 9 Yaniv Kaul 2017-02-13 08:00:54 UTC
Any progress on this?

Comment 10 Lucie Leistnerova 2017-03-08 11:14:33 UTC
Value of used memory in donut corresponds with sum of used memory on hosts.
The used percentage in donut corresponds with the total sum of memory and used memory.

Value of used CPU in donut corresponds with calculation: ((host1 cores * usage) + (host2 cores * usage) ..) / all cores. Changes on the host have affect after dwh reloads (15 minutes).

I did not check over commit. I don't know how it should be calculated and it was not mentioned that this BZ contains it.

tested in ovirt-engine-4.1.1.3-0.1.el7.noarch with ovirt-engine-dashboard-1.1.0-6.el7ev.noarch

So the calculations looks like you suggested but it should be mentioned somewhere. In release notes? The customer could be confused because the tooltips for the calculation are not yet done, see Bug 1358604.

Comment 11 Shirly Radco 2017-04-13 13:14:41 UTC
(In reply to Filip Krepinsky from comment #3)
> Also, should we count total cpu/memory (used in over commit) from all hosts? 
> I think it makes sense to show only over commit for running hosts.

Alexander, please reply how the over commit is currently calculated.
I believe it is calculated after the dwh queries, on the dashboard level.

Comment 12 Alexander Wels 2017-04-13 13:29:49 UTC
Overcommit is calculated like this:

virtual used / physical total * 100

So if I have 100G virtual memory used and my total physical memory is 50G, then my overcommit is 200%.

Or for CPU if I have 100 virtual cores used for VMs, and I have 20 physical cores. Then my over commit is 500%

Note there is a difference between used and total.

Comment 15 Lucie Leistnerova 2017-04-18 13:08:23 UTC
So the calculations on memory and cpu are OK. Over commit is not part of this BZ. If it needs some update, please create new BZ for it.

verified in ovirt-engine-4.1.1.3-0.1.el7.noarch


Note You need to log in before you can comment on or make changes to this bug.