Bug 1518541

Summary: [RFE] Monitor capacity of vdo-enabled brick devices and gluster volumes
Product: [oVirt] ovirt-engine Reporter: Sahina Bose <sabose>
Component: BLL.GlusterAssignee: Denis Chaplygin <dchaplyg>
Status: CLOSED CURRENTRELEASE QA Contact: bipin <bshetty>
Severity: medium Docs Contact:
Priority: high    
Version: 4.1.7.3CC: bshetty, bugs, dchaplyg, fromani, sabose, sasundar, ylavi
Target Milestone: ovirt-4.2.5Keywords: FutureFeature
Target Release: ---Flags: rule-engine: ovirt-4.2+
ylavi: planning_ack+
rule-engine: devel_ack+
sasundar: testing_ack+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ovirt-engine-4.2.5.2 Doc Type: Enhancement
Doc Text:
Feature: Added monitoring of thin storage devices (thin provisioned, compressed or deduplicated) Reason: With thin devices number of available 'free bytes' on storage domain may not be relevant. Storage domain may report that you have a lot of space, while actually you are quite limited. Because of that we need to monitor actually available space and report it. Result: Now we report guaranteed free space (meaning, that you will be able to write at least as much data and may be more) for gluster bricks, bluster volumes and for gluster based storage domains.
 Story Points: ---
Clone Of:
: 1523601 (view as bug list) Environment:
Last Closed: 2018-07-31 15:26:28 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Gluster RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1568893    
Bug Blocks: 1523601    
Attachments:
Description Flags
Cockpit_UI_Verified none

Description Sahina Bose 2017-11-29 07:12:07 UTC 
Description of problem:

VDO thin provisioning will need to be monitored (physical space) ( sysfs)

Alerts (optionally to UI, SMTP, system logs) to be sent that more physical space is needed or the storage admin needs to consider removing unneeded data

proposed Alert levels are:

Notify: that more free space is needed (80%, but not less than 2TB)
Warn: that system is near full (90%, but not less than 1TB)
Critical: the VMs should be paused and / or file system should be put into read-only mode (95%, but not less than 500GB )

ON DEMAND via the UI reporting of space savings / physical space to be included (including exposed in Cockpit)

How reproducible:
NA
Comment 1 Francesco Romani 2018-03-14 14:13:18 UTC 
ovirt 4.2 patches did NOT land in Vdsm 4.20.22. They are expected to be included in the first 4.2.3 build
Comment 2 SATHEESARAN 2018-05-15 11:42:35 UTC 
This feature couldn't be tested as there are few more changes to be fixed.

Denis is trying to build the private rpms to test this feature.

The bug is not reflecting the correct state, so moving this bug to ASSIGNED, until all the patches make it in to the build
Comment 3 Red Hat Bugzilla Rules Engine 2018-05-15 11:42:42 UTC 
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.
Comment 4 Sandro Bonazzola 2018-06-15 09:07:57 UTC 
Sahina, according to comment #2 not all patches are in at 2018-05-15.
No more patches has been added to this bug so this should still have been in ASSIGNED.

What's the real status of this bug? In which release has this bug been fixed?
Moving back to ASSIGNED for visibility.
Comment 5 Sandro Bonazzola 2018-06-15 12:16:20 UTC 
This bug is not marked as blocker and we entered blocker only phase for 4.2.4.
Please consider re-targeting to 4.2.5.
Comment 6 Denis Chaplygin 2018-06-15 15:22:11 UTC 
This one (In reply to SATHEESARAN from comment #2)
> This feature couldn't be tested as there are few more changes to be fixed.
> 


All missing features are in place, this one should be retested.
Comment 9 SATHEESARAN 2018-07-18 06:59:17 UTC 
The dependent bug is moved to ASSIGNED ( currently its in modified ), without that fix, this bug can't be verified.

Moving this bug too to ASSIGNED
Comment 10 SATHEESARAN 2018-07-18 07:00:13 UTC 
Denis,

Could you please move this bug ON_QA along in the fixed-in-version, when the dependent bug BZ 1568893 is moved ON_QA ?
Comment 11 Sandro Bonazzola 2018-07-24 16:10:45 UTC 
(In reply to SATHEESARAN from comment #10)
> Denis,
> 
> Could you please move this bug ON_QA along in the fixed-in-version, when the
> dependent bug BZ 1568893 is moved ON_QA ?

Should be included in ovirt-engine-4.2.5.2
Comment 12 bipin 2018-07-26 06:46:51 UTC 

Verified the bug in ovirt-engine-4.2.5.2.

Steps:
=====
1.Create a vdo based storage domain
2.Create a volume of size 899GB
3.Set the "warning low space indicator" to 98 and "warning low confirmed space indicator" to 97
4.Saw the alerts which it reached that point


Attaching the screenshot

Logs from HE:
============
2018-07-26 11:11:30,848+05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [] EVENT_ID: IRS_DISK_SPACE_LOW_ERROR(201), Critical, Low disk space. vdo_sdd domain has 879 GB of free space.

2018-07-26 11:11:10,692+05 WARN  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [] EVENT_ID: IRS_DISK_SPACE_LOW(26), Warning, Low disk space. vdo_sdd domain has 881 GB of free space.
Comment 13 bipin 2018-07-26 06:47:18 UTC 
Created attachment 1470608 [details]
Cockpit_UI_Verified
Comment 14 SATHEESARAN 2018-07-26 08:24:57 UTC 
(In reply to SATHEESARAN from comment #10)
> Denis,
> 
> Could you please move this bug ON_QA along in the fixed-in-version, when the
> dependent bug BZ 1568893 is moved ON_QA ?

Thanks, this issue is already verified, removing the needinfo
Comment 15 Sandro Bonazzola 2018-07-31 15:26:28 UTC 
This bugzilla is included in oVirt 4.2.5 release, published on July 30th 2018.

Since the problem described in this bug report should be
resolved in oVirt 4.2.5 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.