Bug 1017126

Summary: RFE : Need to revamp logging and alert mechanism to volume level
Product: Red Hat Gluster Storage Reporter: Saurabh <saujain>
Component: glusterdAssignee: Bug Updates Notification Mailing List <rhs-bugs>
Status: CLOSED NOTABUG QA Contact: Saurabh <saujain>
Severity: high Docs Contact:
Priority: low    
Version: 2.1CC: grajaiya, mzywusko, nsathyan, vagarwal, vbellur, vmallika
Target Milestone: ---Keywords: FutureFeature
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-01-12 09:09:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
sosreport of quota1
none
sosreport of quota2
none
sosreport of quota4
none
sosreport of quota3 none

Description Saurabh 2013-10-09 09:49:46 UTC
Description of problem:

changed the value of default-soft-limit to 70% and result is not as expected, as the "A" msg comes in some brick and not is some. 

As well as it appears after 90% of limit is crossed.

Volume Name: dist-rep1
Type: Distributed-Replicate
Volume ID: d5b1c9c6-d067-4e47-ae45-08425faca23d
Status: Started
Number of Bricks: 6 x 2 = 12
Transport-type: tcp
Bricks:
Brick1: 10.70.42.186:/rhs/brick1/d1r11
Brick2: 10.70.43.181:/rhs/brick1/d1r21
Brick3: 10.70.43.18:/rhs/brick1/d2r11
Brick4: 10.70.43.22:/rhs/brick1/d2r21
Brick5: 10.70.42.186:/rhs/brick1/d3r11
Brick6: 10.70.43.181:/rhs/brick1/d3r21
Brick7: 10.70.43.18:/rhs/brick1/d4r11
Brick8: 10.70.43.22:/rhs/brick1/d4r21
Brick9: 10.70.42.186:/rhs/brick1/d5r11
Brick10: 10.70.43.181:/rhs/brick1/d5r21
Brick11: 10.70.43.18:/rhs/brick1/d6r11
Brick12: 10.70.43.22:/rhs/brick1/d6r21
Options Reconfigured:
features.default-soft-limit: 70%
server.root-squash: disable
features.quota-deem-statfs: on
features.quota: on



Version-Release number of selected component (if applicable):
glusterfs-3.4.0.34rhs-1.el6rhs.x86_64

How reproducible:
always

Steps to Reproduce:
1. change the defaulf-soft-limit to 70%
2. create a subdir and set a limit of 100MB
3. now start creating data in it

This time I created subdir6 and started creating data in it.
[root@quota2 ~]# gluster volume quota dist-rep1 list
                  Path                   Hard-limit Soft-limit   Used  Available
--------------------------------------------------------------------------------
/                                         10.0GB       70%       2.3GB   7.7GB
/dir1                                      4.0GB       70%       2.3GB   1.7GB
/dir1/subdir1                            100.0MB       70%     101.0MB  0Bytes
/dir1/subdir2                            100.0MB       70%     101.0MB  0Bytes
/dir1/subdir4                            100.0MB       70%     100.0MB  0Bytes
/dir1/subdir5                            100.0MB       70%     100.0MB  0Bytes
/dir1/subdir6                            100.0MB       70%     101.0MB  0Bytes


Actual results:
/rhs/brick1/d1r11 --- nologs about Alert messge for subdir6
/rhs/brick1/d2r11 --- nologs about Alert messge for subdir6
/rhs/brick1/d3r11 --- alert message is "Usage is above soft limit: 97.0MB used by /dir1//subdir6/"
/rhs/brick1/d4r11 --- alert message is "Usage is above soft limit: 91.0MB used by /dir1//subdir6/"
/rhs/brick1/d5r11 --- alert message is "Usage is above soft limit: 92.0MB used by /dir1//subdir6/"
/rhs/brick1/d6r11 --- nologs about Alert messge for subdir6

This is big confusion, 
1. where to find the "A" message. 
2. then when the default-soft-limit is set to 70% and it is appearing after 90% is wrong.

Expected results:

avoid all confusions raised in Acutal Result section

Additional info:

Comment 2 Saurabh 2013-10-10 11:07:05 UTC
Created attachment 810456 [details]
sosreport of quota1

Comment 3 Saurabh 2013-10-10 11:10:28 UTC
Created attachment 810458 [details]
sosreport of quota2

Comment 4 Saurabh 2013-10-10 11:12:16 UTC
Created attachment 810459 [details]
sosreport of quota4

Comment 5 Saurabh 2013-10-10 11:14:42 UTC
Created attachment 810460 [details]
sosreport of quota3

Comment 8 Vivek Agarwal 2013-10-17 07:38:59 UTC
Per discussion with Shanks and Saurabh, need to revamp the logging mechanism to reflect volume ops

Comment 9 Vivek Agarwal 2014-02-20 08:36:41 UTC
adding 3.0 flag and removing 2.1.z

Comment 11 Vijaikumar Mallikarjuna 2015-01-12 09:09:10 UTC
Alert message seeing in some bricks and not seeing in some bricks depends on which brick data is flowing.
This is a expected behavior, so closing as NOTABUG