Bug 1085254

Summary: Quota: Soft limit crosed alert not logged in brick or syslogs
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Sahina Bose <sabose>
Component: glusterfsAssignee: Nagaprasad Sathyanarayana <nsathyan>
Status: CLOSED ERRATA QA Contact: Saurabh <saujain>
Severity: high Docs Contact:
Priority: high    
Version: rhgs-3.0CC: asrivast, dpati, mzywusko, nsathyan, psriniva, rgowdapp, smohan, ssamanta
Target Milestone: ---   
Target Release: RHGS 3.0.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.6.0.25-1 Doc Type: Bug Fix
Doc Text:
Previously, warning messages were not logged when quota soft limit was met. With this fix, setting the quota soft-timeout and hard-timeout values to zero ensures logging of warning messages.
Story Points: ---
Clone Of:
: 1087198 (view as bug list) Environment:
Last Closed: 2014-09-22 19:35:17 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1087198    

Description Sahina Bose 2014-04-08 08:56:20 UTC
Description of problem:

Once soft limit / hard limit is crossed, there are no "Alert" level logs indicating that usage is above soft-limit.

The only log messages are ones similar to these:
[2014-04-06 14:48:33.512884] I [server-rpc-fops.c:1447:server_writev_cbk] 0-test-vol-server: 634: WRITEV 0 (0f74aac5-eba2-4897-919a-6f8ada619a14) ==> (Disk quota exceeded)
[2014-04-06 14:48:33.512979] I [server-rpc-fops.c:1447:server_writev_cbk] 0-test-vol-server: 635: WRITEV 0 (0f74aac5-eba2-4897-919a-6f8ada619a14) ==> (Disk quota exceeded)

The alert message was seen only when the volume quota option "alert-time" was set to 0.



Version-Release number of selected component (if applicable):
RHS 2.1 Update 2

How reproducible:


Steps to Reproduce:
1. Create a volume, mount the volume and create a dir
2. Set limit on the dir for the volume
3. Add files using the mounted client, until quota limit set is exceeded.

Actual results:
No alert message

Expected results:
Messages are logged when the limit is crossed

Additional info:
The alert message is required for the Nagios log monitoring so that the Nagios volume service state can be changed when quota limits are crossed

Comment 1 Dusmant 2014-04-11 10:44:19 UTC
This log message is very critical for the quota monitoring feature in RHSC. So, i have marked the bug as a development blocker and also put the severity at High.

Comment 3 Nagaprasad Sathyanarayana 2014-04-28 07:11:10 UTC
Quoting from Varun's e-mail on this BZ.

======================================================
In the meantime to not to block the testing and development of the dependency features you can use the below two options of Quota.

gluster volume quota <volname> hard-timeout 0
gluster volume quota <volname> soft-timeout 0
gluster volume quota <volname> alert-time 0 <or-a-very-minimal-value-in-the-order-of-seconds>

Upon setting these configurations, quota never skips the logging.
======================================================

Comment 4 Sahina Bose 2014-04-28 08:31:43 UTC
Do setting these values have any performance impact?

Comment 5 vpshastry 2014-04-29 06:35:21 UTC
Yes, these settings may reduce the performance a bit. But this is the only way to ensure the logging in all the cases.

Comment 6 Nagaprasad Sathyanarayana 2014-05-19 10:56:19 UTC
Setting flags required to add BZs to RHS 3.0 Errata

Comment 10 vpshastry 2014-07-11 09:35:17 UTC
The logging is done after the check for the soft limit cross on every write. Without the quota size updation from all the bricks, quota doesn't know whether the soft limit has crossed. This is a problem of trade off between correctness vs performance. As suggested in the comment #3 for the accurate logging one should tune the soft timeout and hard timeout to zero.

Comment 13 Pavithra 2014-08-06 09:49:00 UTC
Hi Varun,

Please review the edited doc text for technical accuracy and sign off.

Comment 15 Saurabh 2014-09-19 06:59:36 UTC
based on the steps provided in comment#3 of this BZ, the alert messages are seen even after limit are reached. But there are many alert messages now. 

Since, this BZ is only about getting alert messages in logs after limit have reached, hence moving to verified.

Comment 17 errata-xmlrpc 2014-09-22 19:35:17 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2014-1278.html

Comment 18 Red Hat Bugzilla 2023-09-14 02:06:00 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days