Bug 1336896

Summary: Storage utilization: negative available space is reported after removing one storage
Product: [oVirt] ovirt-engine-dashboard Reporter: Pavel Novotny <pnovotny>
Component: CoreAssignee: Alexander Wels <awels>
Status: CLOSED CURRENTRELEASE QA Contact: Pavel Novotny <pnovotny>
Severity: high Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: awels, lsvaty, oourfali, vszocs
Target Milestone: ovirt-4.0.0-rcFlags: rule-engine: ovirt-4.0.0+
rule-engine: planning_ack+
rule-engine: devel_ack+
pnovotny: testing_ack+
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-08-04 13:28:29 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: UX RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Negative storage utilization after one storage removal none

Description Pavel Novotny 2016-05-17 17:59:18 UTC
Created attachment 1158428 [details]
Negative storage utilization after one storage removal

Description of problem:
After removing one of two storages, the available space changes to negative number.


Version-Release number of selected component (if applicable):
ovirt-engine-dashboard-0.0.1-0.0.20160511git4debbb1.el7.centos.noarch

How reproducible:
100%

Steps to Reproduce:
1. Add two storages to engine, each having:
   Size: 767 GB, Available: 215 GB, Used: 552 GB
   (hint: both storages reside on the same NFS server volume so they report same values)
2. Dashboard Storage utilization will report:
   0.4 Available of 1.5 TB (= 2x 215 GB of 2x 767 GB); this is correct
3. Remove one storage.
4. Check again updated Dashboard Storage utilization.

Actual results:
See attached screenshot.
Storage utilization is now: -0.3 Available of 0.7 TB

Expected results:
0.2 Available of 0.7 TB

Additional info:

A guess the computation is:
0.4 Available before removal - 0.7 *storage size* to be removed = -0.3 Available

but should be:

0.4 Available before removal - 0.2 Available to be removed = 0.2 Available

Comment 1 Pavel Novotny 2016-05-17 18:29:01 UTC
Update: After cca. 15 minutes (after DWH data pull?), the reported Storage utilization corrected itself to "0.2 Available of 0.7 TB".
So the negative reported value is just temporary, however it is pretty glaring in the UI.

Comment 2 Alexander Wels 2016-05-23 16:31:02 UTC
I believe this is a problem with the DWH that has been fixed. Updating the packages caused another issue with the DWH which has also been fixed. So hopefully once the new build is ready for this, that will also fix the problem in the dashboard.

Comment 3 Pavel Novotny 2016-05-24 13:50:47 UTC
Confirming that previous DWH was not working properly, which most probably caused the wrong available space to be reported in Dashboard.

After updating to [1] I don't see errors in DWH log any more.
But after 2 hours of running with functioning DWH, Dashboard still reports the same negative available space. 
@Alex, is it ok? Do I need to wait longer time before the value gets corrected?


[1] ovirt-engine-dwh-4.0.0-0.0.master.20160523112155.gitb69e9ab.el7.centos.noarch
    ovirt-engine-4.0.0-0.0.master.20160523171412.git1a7a1f3.el7.centos.noarch

Comment 4 Alexander Wels 2016-05-24 14:53:48 UTC
So I think I figured out the difference between what you did and what I did. You detached the storage domain, but didn't remove it. I did. The query didn't take the status of the detached storage domain into account. I am going to update the queries to do so.

Comment 5 Pavel Novotny 2016-08-02 16:38:26 UTC
rhevm-4.0.2.3-0.1.el7ev.noarch
ovirt-engine-dashboard-1.0.1-0.el7ev.x86_64

The negative value for available space is no longer reported.
But now there is still a time window, 
where after removing or just detaching a 2nd storage domain,
the available space is incorrectly reported as 0 TiB. 
However, the zero value is corrected after few minutes.

Environment:
2 NFS storages, each one has: Size 767 GB, Available: 224 GB, Used: 543 GB

Take #1:
* only 1 storage is present: 
  Dashboard @ 4:53:13 PM:  0.2 Available of 0.7 TiB / 0.5 TiB Used
* 2nd storage added: 
  Dashboard @ 4:57:58 PM: 0.4 Available of 1.5 TiB / 1.1 TiB Used
* 2nd storage detached (but not removed):
  Dashboard @ 5:03:05 PM: 0 Available of 0.7 TiB / 0.7 TiB Used  <<-- WRONG
  Dashboard @ 5:27:03 PM: 0.2 Available of 0.7 TiB / 0.5 TiB Used  <<-- CORRECT

Take #2:
* 2nd storage added: 
  5:38:53 PM: 0.4 Available of 1.5 TiB / 1.1 TiB Used
* 2nd storage detached and removed:
  5:43:59 PM - 0 Available of 0.7 TiB / 0.7 TiB Used  <<-- WRONG
  5:50:46 PM - 0.2 Available of 0.7 TiB / 0.5 TiB Used  <<-- CORRECT

Take #3:
* 2nd storage added: 
  5:57:38 PM: 0.4 Available of 1.5 TiB / 1.1 TiB Used
* 2nd storage detached (but not removed):
  6:01:57 PM - 0 Available of 0.7 TiB / 0.7 TiB Used  <<--WRONG
  6:14:10 PM - 0.2 Available of 0.7 TiB / 0.5 TiB Used  <<-- CORRECT

Comment 6 Lukas Svaty 2016-08-03 07:19:04 UTC
Moving to ASSIGNED FailedQA as for comment #5. Can we get a better update interval or remove the middle part  of  0 available and update after 15 minutes to correct result?

Comment 7 Red Hat Bugzilla Rules Engine 2016-08-03 07:19:07 UTC
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Comment 8 Oved Ourfali 2016-08-03 07:50:56 UTC
Iirc we have a fix not to show negative values. 
Vojtech, you pushed that, right?

Comment 9 Alexander Wels 2016-08-03 12:21:44 UTC
Oved correct that should be in 1.0.2-1 however that is not the 'failed' condition. The failed condition is that it is reporting 0 available when in fact there is some available, it just hasn't gotten the updated data yet.

Comment 10 Oved Ourfali 2016-08-03 12:27:35 UTC
I see.
As we collect DWH data only every 15 minutes, I don't see how we can fix that. Only the inventory part consist of "real time data". The rest aren't.
Moving back to ON_QA.
Another bug should be opened on the other issue, if any (as we might need to close this as wontfix).

Comment 11 Lukas Svaty 2016-08-03 13:36:48 UTC
I agree verifying this one as it is not reporting negative values anymore, will open new.

Comment 12 Vojtech Szocs 2016-08-04 13:53:25 UTC
(In reply to Oved Ourfali from comment #8)
> Iirc we have a fix not to show negative values. 
> Vojtech, you pushed that, right?

Sorry for late response. Yes, this should be fixed in 4.0.2 (dashboard 1.0.2).