Bug 1518541 - [RFE] Monitor capacity of vdo-enabled brick devices and gluster volumes
Summary: [RFE] Monitor capacity of vdo-enabled brick devices and gluster volumes
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Gluster
Version: 4.1.7.3
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: ovirt-4.2.5
: ---
Assignee: Denis Chaplygin
QA Contact: bipin
URL:
Whiteboard:
Depends On: 1568893
Blocks: 1523601
TreeView+ depends on / blocked
 
Reported: 2017-11-29 07:12 UTC by Sahina Bose
Modified: 2018-07-31 15:26 UTC (History)
7 users (show)

Fixed In Version: ovirt-engine-4.2.5.2
Doc Type: Enhancement
Doc Text:
Feature: Added monitoring of thin storage devices (thin provisioned, compressed or deduplicated) Reason: With thin devices number of available 'free bytes' on storage domain may not be relevant. Storage domain may report that you have a lot of space, while actually you are quite limited. Because of that we need to monitor actually available space and report it. Result: Now we report guaranteed free space (meaning, that you will be able to write at least as much data and may be more) for gluster bricks, bluster volumes and for gluster based storage domains.
Clone Of:
: 1523601 (view as bug list)
Environment:
Last Closed: 2018-07-31 15:26:28 UTC
oVirt Team: Gluster
Embargoed:
rule-engine: ovirt-4.2+
ylavi: planning_ack+
rule-engine: devel_ack+
sasundar: testing_ack+

Attachments (Terms of Use)
Cockpit_UI_Verified (73.35 KB, image/png)
2018-07-26 06:47 UTC, bipin 		no flags Details
View All


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 87534 0 master MERGED gluster: Added reporting local LVM logical volumes info 2020-01-28 21:57:09 UTC
oVirt gerrit 87588 0 master MERGED gluster: Added reporting local LVM physical volumes info 2020-01-28 21:57:09 UTC
oVirt gerrit 87690 0 master MERGED gluster: Added reporting VDO volumes info 2020-01-28 21:57:09 UTC
oVirt gerrit 88050 0 master MERGED gluster: Refactored refreshVolumeDetails 2020-01-28 21:57:09 UTC
oVirt gerrit 88051 0 master MERGED gluster: Added gathering local LVM logical volumes info 2020-01-28 21:57:09 UTC
oVirt gerrit 88052 0 master MERGED gluster: Added gathering local LVM physical volumes info 2020-01-28 21:57:09 UTC
oVirt gerrit 88053 0 master MERGED gluster: Added reporting VDO volumes info 2020-01-28 21:57:10 UTC
oVirt gerrit 88054 0 master MERGED gluster: Added local volumes information retrieval. 2020-01-28 21:57:10 UTC
oVirt gerrit 88055 0 master MERGED gluster: Added brick size calculation. 2020-01-28 21:57:10 UTC
oVirt gerrit 88203 0 master MERGED gluster: Added storing brick's confirmed free size 2020-01-28 21:57:10 UTC
oVirt gerrit 88357 0 master MERGED gluster: Added calculation and storing of the volume confirmed free size. 2020-01-28 21:57:10 UTC
oVirt gerrit 88592 0 master MERGED gluster:Added brick confirmed size monitoring 2020-01-28 21:57:10 UTC
oVirt gerrit 88697 0 master MERGED gluster:Added volume confirmed size monitoring 2020-01-28 21:57:11 UTC
oVirt gerrit 88892 0 master ABANDONED gluster: Adding conditional vdo dependency 2020-01-28 21:57:11 UTC
oVirt gerrit 88921 0 ovirt-4.2 MERGED gluster: Added reporting local LVM logical volumes info 2020-01-28 21:57:11 UTC
oVirt gerrit 88922 0 ovirt-4.2 MERGED gluster: Added reporting local LVM physical volumes info 2020-01-28 21:57:11 UTC
oVirt gerrit 88923 0 ovirt-4.2 MERGED gluster: Added reporting VDO volumes info 2020-01-28 21:57:11 UTC
oVirt gerrit 89057 0 master ABANDONED gluster:Added storing volume's confirmed free to the SD 2020-01-28 21:57:11 UTC
oVirt gerrit 89058 0 master MERGED webadmin: Added confirmed free size to the storage domain view 2020-01-28 21:57:12 UTC
oVirt gerrit 89760 0 ovirt-engine-4.2 MERGED gluster: Refactored refreshVolumeDetails 2020-01-28 21:57:12 UTC
oVirt gerrit 89761 0 ovirt-engine-4.2 MERGED gluster: Added gathering local LVM logical volumes info 2020-01-28 21:57:12 UTC
oVirt gerrit 89776 0 ovirt-engine-4.2 MERGED gluster: Added gathering local LVM physical volumes info 2020-01-28 21:57:12 UTC
oVirt gerrit 89777 0 ovirt-engine-4.2 MERGED gluster: Added reporting VDO volumes info 2020-01-28 21:57:12 UTC
oVirt gerrit 89778 0 ovirt-engine-4.2 MERGED gluster: Added brick size calculation. 2020-01-28 21:57:12 UTC
oVirt gerrit 89779 0 ovirt-engine-4.2 MERGED gluster: Added local volumes information retrieval. 2020-01-28 21:57:13 UTC
oVirt gerrit 89780 0 ovirt-engine-4.2 MERGED gluster: Added storing brick's confirmed free size 2020-01-28 21:57:13 UTC
oVirt gerrit 89865 0 ovirt-engine-4.2 MERGED gluster:Added brick confirmed size monitoring 2020-01-28 21:57:13 UTC
oVirt gerrit 89866 0 ovirt-engine-4.2 MERGED gluster: Added calculation and storing of the volume confirmed free size. 2020-01-28 21:57:13 UTC
oVirt gerrit 89867 0 ovirt-engine-4.2 MERGED gluster:Added volume confirmed size monitoring 2020-01-28 21:57:13 UTC
oVirt gerrit 89868 0 ovirt-engine-4.2 MERGED webadmin: Added confirmed free size to the storage domain view 2020-01-28 21:57:13 UTC

Description Sahina Bose 2017-11-29 07:12:07 UTC 
Description of problem:

VDO thin provisioning will need to be monitored (physical space) ( sysfs)

Alerts (optionally to UI, SMTP, system logs) to be sent that more physical space is needed or the storage admin needs to consider removing unneeded data

proposed Alert levels are:

Notify: that more free space is needed (80%, but not less than 2TB)
Warn: that system is near full (90%, but not less than 1TB)
Critical: the VMs should be paused and / or file system should be put into read-only mode (95%, but not less than 500GB )

ON DEMAND via the UI reporting of space savings / physical space to be included (including exposed in Cockpit)

How reproducible:
NA
Comment 1 Francesco Romani 2018-03-14 14:13:18 UTC 
ovirt 4.2 patches did NOT land in Vdsm 4.20.22. They are expected to be included in the first 4.2.3 build
Comment 2 SATHEESARAN 2018-05-15 11:42:35 UTC 
This feature couldn't be tested as there are few more changes to be fixed.

Denis is trying to build the private rpms to test this feature.

The bug is not reflecting the correct state, so moving this bug to ASSIGNED, until all the patches make it in to the build
Comment 3 Red Hat Bugzilla Rules Engine 2018-05-15 11:42:42 UTC 
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.
Comment 4 Sandro Bonazzola 2018-06-15 09:07:57 UTC 
Sahina, according to comment #2 not all patches are in at 2018-05-15.
No more patches has been added to this bug so this should still have been in ASSIGNED.

What's the real status of this bug? In which release has this bug been fixed?
Moving back to ASSIGNED for visibility.
Comment 5 Sandro Bonazzola 2018-06-15 12:16:20 UTC 
This bug is not marked as blocker and we entered blocker only phase for 4.2.4.
Please consider re-targeting to 4.2.5.
Comment 6 Denis Chaplygin 2018-06-15 15:22:11 UTC 
This one (In reply to SATHEESARAN from comment #2)
> This feature couldn't be tested as there are few more changes to be fixed.
> 


All missing features are in place, this one should be retested.
Comment 9 SATHEESARAN 2018-07-18 06:59:17 UTC 
The dependent bug is moved to ASSIGNED ( currently its in modified ), without that fix, this bug can't be verified.

Moving this bug too to ASSIGNED
Comment 10 SATHEESARAN 2018-07-18 07:00:13 UTC 
Denis,

Could you please move this bug ON_QA along in the fixed-in-version, when the dependent bug BZ 1568893 is moved ON_QA ?
Comment 11 Sandro Bonazzola 2018-07-24 16:10:45 UTC 
(In reply to SATHEESARAN from comment #10)
> Denis,
> 
> Could you please move this bug ON_QA along in the fixed-in-version, when the
> dependent bug BZ 1568893 is moved ON_QA ?

Should be included in ovirt-engine-4.2.5.2
Comment 12 bipin 2018-07-26 06:46:51 UTC 

Verified the bug in ovirt-engine-4.2.5.2.

Steps:
=====
1.Create a vdo based storage domain
2.Create a volume of size 899GB
3.Set the "warning low space indicator" to 98 and "warning low confirmed space indicator" to 97
4.Saw the alerts which it reached that point


Attaching the screenshot

Logs from HE:
============
2018-07-26 11:11:30,848+05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [] EVENT_ID: IRS_DISK_SPACE_LOW_ERROR(201), Critical, Low disk space. vdo_sdd domain has 879 GB of free space.

2018-07-26 11:11:10,692+05 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [] EVENT_ID: IRS_DISK_SPACE_LOW(26), Warning, Low disk space. vdo_sdd domain has 881 GB of free space.
Comment 13 bipin 2018-07-26 06:47:18 UTC 
Created attachment 1470608 [details]
Cockpit_UI_Verified
Comment 14 SATHEESARAN 2018-07-26 08:24:57 UTC 
(In reply to SATHEESARAN from comment #10)
> Denis,
> 
> Could you please move this bug ON_QA along in the fixed-in-version, when the
> dependent bug BZ 1568893 is moved ON_QA ?

Thanks, this issue is already verified, removing the needinfo
Comment 15 Sandro Bonazzola 2018-07-31 15:26:28 UTC 
This bugzilla is included in oVirt 4.2.5 release, published on July 30th 2018.

Since the problem described in this bug report should be
resolved in oVirt 4.2.5 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.
Note You need to log in before you can comment on or make changes to this bug.