Bug 1023149
| Summary: | Quota limit set to higher value doesn't comes to effect after remove-brick | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Gluster Storage | Reporter: | shylesh <shmohan> |
| Component: | quota | Assignee: | Vijaikumar Mallikarjuna <vmallika> |
| Status: | CLOSED WORKSFORME | QA Contact: | storage-qa-internal <storage-qa-internal> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 2.1 | CC: | rhs-bugs, shmohan, smohan, spalai, storage-qa-internal, vagarwal, vbellur, vmallika |
| Target Milestone: | --- | Keywords: | ZStream |
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2015-11-17 08:50:03 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Tried to reproduce the issue.
[root@vm1 dir2]# glusterfs --version
glusterfs 3.4.0.56rhs built on Jan 10 2014 22:58:18
[root@vm1 ~]# gvi
Volume Name: test1
Type: Distributed-Replicate
Volume ID: 7c8fa6e4-3e43-4fe0-a30f-6e8c3b5e87b4
Status: Created
Number of Bricks: 4 x 2 = 8
Transport-type: tcp
Bricks:
Brick1: 192.168.122.240:/brick1/1
Brick2: 192.168.122.240:/brick2/2
Brick3: 192.168.122.240:/brick1/3
Brick4: 192.168.122.240:/brick2/4
Brick5: 192.168.122.240:/brick1/5
Brick6: 192.168.122.240:/brick2/6
Brick7: 192.168.122.240:/brick1/7
Brick8: 192.168.122.240:/brick2/8
[root@vm1 test]# gluster volume quota test1 list
Path Hard-limit Soft-limit Used Available Soft-limit exceeded? Hard-limit exceeded?
---------------------------------------------------------------------------------------------------------------------------
/test 500.0MB 80% 0Bytes 500.0MB No No
/root 500.0MB 80% 0Bytes 500.0MB No No
Used the following script to create files
[root@vm1 dir1]# for i in {1..600} ; do dd if=/dev/zero of=file$i bs=1MB count=1 ; done
[root@vm1 dir1]# gluster volume quota test1 list
Path Hard-limit Soft-limit Used Available Soft-limit exceeded? Hard-limit exceeded?
---------------------------------------------------------------------------------------------------------------------------
/test 500.0MB 80% 572.5MB 0Bytes Yes Yes
/root 500.0MB 80% 0Bytes 500.0MB No No
[root@vm1 dir1]# gluster volume remove-brick test1 192.168.122.240:/brick1/7 192.168.122.240:/brick2/8 status
Node Rebalanced-files size scanned failures skipped status run time in secs
--------- ----------- ----------- ----------- ----------- ----------- ------------ --------------
localhost 298 284.2MB 600 0 0 completed 62.00
* set limit of test dir to 1GB
[root@vm1 dir1]# gluster volume quota test1 list
Path Hard-limit Soft-limit Used Available Soft-limit exceeded? Hard-limit exceeded?
---------------------------------------------------------------------------------------------------------------------------
/test 1.0GB 80% 572.5MB 451.5MB No No
/root 500.0MB 80% 0Bytes 500.0MB No No
[root@vm1 dir1]#
Created a directory /test/dir2 and created files under it. But it filled the directory quota limit completely without any error.
[root@vm1 dir2]# gluster volume quota test1 list
Path Hard-limit Soft-limit Used Available Soft-limit exceeded? Hard-limit exceeded?
---------------------------------------------------------------------------------------------------------------------------
/test 1.0GB 80% 1.0GB 0Bytes Yes Yes
/root 500.0MB 80% 0Bytes 500.0MB No No
[root@vm1 dir2]#
Shylesh can you verify the bug in latest build.
Not able to re-create the problem as per comment# 3. Also 2.1 is EOL'ed, so closing the bug The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days |
Description of problem: After starting the remove-brick if we change the quota limit to some higher value doesn't comes into effect and old quota limit will be still imposed. Version-Release number of selected component (if applicable): 3.4.0.36rhs-1.el6rhs.x86_64 How reproducible: always Steps to Reproduce: 1.I had a 16x2 distributed-replicate volume 2.creates sub directories root and test 3. set the quota limit on both directories as 1GB [root@rhs-client4 test]# gluster v quota s1 list Path Hard-limit Soft-limit Used Available -------------------------------------------------------------------------------- /root 1.0GB 80% 1005.2MB 18.8MB /test 1.0GB 80% 1.0GB 0MB 4. created a subdirectory inside test and filled up with files till i get "Disk quota exceeded" 5. started removing one of the pair using remove-brick start option gluster v remove-brick s1 rhs-client9.lab.eng.blr.redhat.com:/home/s130 rhs-client39.lab.eng.blr.redhat.com:/home/s131 start 6. once the status is completed , without committing changed the quota limit on test directory to a higher value [root@rhs-client4 test]# gluster v quota s1 list Path Hard-limit Soft-limit Used Available -------------------------------------------------------------------------------- /root 1.0GB 80% 1005.2MB 18.8MB /test 2.0GB 80% 1.0GB 1.0GB 7. now create another directory inside /test say test/dir1 8. try to create files inside this directory 9. some files got created and some files not , they failed with "Disk quota exceeded" Actual results: Expected results: New quota value should take effect and data creation should be possible since we have increased the quota limit Additional info: cluster info ------------ rhs-client4.lab.eng.blr.redhat.com rhs-client40.lab.eng.blr.redhat.com rhs-client39.lab.eng.blr.redhat.com rhs-client9.lab.eng.blr.redhat.com mount point ============= rhs-client4.lab.eng.blr.redhat.com:/mnt mount type ---------- NFS Volume Name: s1 Type: Distributed-Replicate Volume ID: 57f48f8a-d3d3-4e7c-8ea1-0240a17e37f0 Status: Started Number of Bricks: 15 x 2 = 30 Transport-type: tcp Bricks: Brick1: rhs-client4.lab.eng.blr.redhat.com:/home/s10 Brick2: rhs-client9.lab.eng.blr.redhat.com:/home/s11 Brick3: rhs-client39.lab.eng.blr.redhat.com:/home/s12 Brick4: rhs-client4.lab.eng.blr.redhat.com:/home/s13 Brick5: rhs-client9.lab.eng.blr.redhat.com:/home/s14 Brick6: rhs-client39.lab.eng.blr.redhat.com:/home/s15 Brick7: rhs-client4.lab.eng.blr.redhat.com:/home/s16 Brick8: rhs-client9.lab.eng.blr.redhat.com:/home/s17 Brick9: rhs-client39.lab.eng.blr.redhat.com:/home/s18 Brick10: rhs-client4.lab.eng.blr.redhat.com:/home/s19 Brick11: rhs-client9.lab.eng.blr.redhat.com:/home/s110 Brick12: rhs-client39.lab.eng.blr.redhat.com:/home/s111 Brick13: rhs-client4.lab.eng.blr.redhat.com:/home/s112 Brick14: rhs-client9.lab.eng.blr.redhat.com:/home/s113 Brick15: rhs-client9.lab.eng.blr.redhat.com:/home/s114 Brick16: rhs-client39.lab.eng.blr.redhat.com:/home/s115 Brick17: rhs-client4.lab.eng.blr.redhat.com:/home/s116 Brick18: rhs-client9.lab.eng.blr.redhat.com:/home/s117 Brick19: rhs-client39.lab.eng.blr.redhat.com:/home/s118 Brick20: rhs-client4.lab.eng.blr.redhat.com:/home/s119 Brick21: rhs-client4.lab.eng.blr.redhat.com:/home/s120 Brick22: rhs-client9.lab.eng.blr.redhat.com:/home/s121 Brick23: rhs-client9.lab.eng.blr.redhat.com:/home/s122 Brick24: rhs-client39.lab.eng.blr.redhat.com:/home/s123 Brick25: rhs-client39.lab.eng.blr.redhat.com:/home/s124 Brick26: rhs-client4.lab.eng.blr.redhat.com:/home/s125 Brick27: rhs-client39.lab.eng.blr.redhat.com:/home/s126 Brick28: rhs-client4.lab.eng.blr.redhat.com:/home/s127 Brick29: rhs-client4.lab.eng.blr.redhat.com:/home/s128 Brick30: rhs-client9.lab.eng.blr.redhat.com:/home/s129 Options Reconfigured: features.quota: on [root@rhs-client4 an]# gluster v quota s1 list Path Hard-limit Soft-limit Used Available -------------------------------------------------------------------------------- /root 1.0GB 80% 1005.2MB 18.8MB /test 2.0GB 80% 1.1GB 945.2MB though /test has 1more GB to run out of quota , data creation still fails 1048576 bytes (1.0 MB) copied, 0.208439 s, 5.0 MB/s 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.207601 s, 5.1 MB/s dd: opening `97': Disk quota exceeded 1+0 records in 1+0 records out 1048576 bytes (1.0 MB) copied, 0.209932 s, 5.0 MB/s 1+0 records in 1+0 records out 2013-10-24 17:06:11.251030] W [client-rpc-fops.c:1579:client3_3_finodelk_cbk] 0-s1-client-20: remote operation failed: No such file or directory [2013-10-24 17:06:11.251843] W [client-rpc-fops.c:1579:client3_3_finodelk_cbk] 0-s1-client-20: remote operation failed: No such file or directory [2013-10-24 17:06:11.252447] W [client-rpc-fops.c:1579:client3_3_finodelk_cbk] 0-s1-client-20: remote operation failed: No such file or directory [2013-10-24 17:06:11.252617] W [client-rpc-fops.c:1579:client3_3_finodelk_cbk] 0-s1-client-20: remote operation failed: No such file or directory [2013-10-24 17:06:11.253111] W [client-rpc-fops.c:1579:client3_3_finodelk_cbk] 0-s1-client-20: remote operation failed: No such file or directory [2013-10-24 17:06:11.255517] W [client-rpc-fops.c:928:client3_3_flush_cbk] 0-s1-client-20: remote operation failed: No such file or directory [2013-10-24 17:06:11.261413] W [client-rpc-fops.c:2058:client3_3_create_cbk] 0-s1-client-25: remote operation failed: Disk quota exceeded. Path: <gfid:57d9 3e36-2c38-4029-83e4-960de5779bd0>/an/86 [2013-10-24 17:06:11.261487] W [client-rpc-fops.c:2058:client3_3_create_cbk] 0-s1-client-24: remote operation failed: Disk quota exceeded. Path: <gfid:57d9 3e36-2c38-4029-83e4-960de5779bd0>/an/86 [2013-10-24 17:06:11.261515] W [nfs3.c:2360:nfs3svc_create_cbk] 0-nfs: 88ab8c3b: <gfid:57d93e36-2c38-4029-83e4-960de5779bd0>/an/86 => -1 (Disk quota exceed ed) [2013-10-24 17:06:11.267027] W [client-rpc-fops.c:2058:client3_3_create_cbk] 0-s1-client-11: remote operation failed: Disk quota exceeded. Path: <gfid:57d9 3e36-2c38-4029-83e4-960de5779bd0>/an/87 [2013-10-24 17:06:11.432273] W [client-rpc-fops.c:1579:client3_3_finodelk_cbk] 0-s1-client-11: remote operation failed: No such file or directory [2013-10-24 17:06:11.432905] W [client-rpc-fops.c:1579:client3_3_finodelk_cbk] 0-s1-client-11: remote operation failed: No such file or directory [2013-10-24 17:06:11.433680] W [client-rpc-fops.c:1579:client3_3_finodelk_cbk] 0-s1-client-11: remote operation failed: No such file or directory [2013-10-24 17:06:11.434305] W [client-rpc-fops.c:1579:client3_3_finodelk_cbk] 0-s1-client-11: remote operation failed: No such file or directory [2013-10-24 17:06:11.434695] W [client-rpc-fops.c:1579:client3_3_finodelk_cbk] 0-s1-client-11: remote operation failed: No such file or directory [2013-10-24 17:06:11.435108] W [client-rpc-fops.c:1579:client3_3_finodelk_cbk] 0-s1-client-11: remote operation failed: No such file or directory [2013-10-24 17:06:11.435571] W [client-rpc-fops.c:1579:client3_3_finodelk_cbk] 0-s1-client-11: remote operation failed: No such file or directory [2013-10-24 17:06:11.436091] W [client-rpc-fops.c:1579:client3_3_finodelk_cbk] 0-s1-client-11: remote operation failed: No such file or directory [2013-10-24 17:06:11.436445] W [client-rpc-fops.c:1579:client3_3_finodelk_cbk] 0-s1-client-11: remote operation failed: No such file or directory [2013-10-24 17:06:11.436715] W [client-rpc-fops.c:1579:client3_3_finodelk_cbk] 0-s1-client-11: remote operation failed: No such file or directory [2013-10-24 17:06:11.436847] W [client-rpc-fops.c:1579:client3_3_finodelk_cbk] 0-s1-client-11: remote operation failed: No such file or directory [2013-10-24 17:06:11.436963] W [client-rpc-fops.c:1579:client3_3_finodelk_cbk] 0-s1-client-11: remote operation failed: No such file or directory [2013-10-24 17:06:11.437074] W [client-rpc-fops.c:1579:client3_3_finodelk_cbk] 0-s1-client-11: remote operation failed: No such file or directory [2013-10-24 17:06:11.437214] W [client-rpc-fops.c:1579:client3_3_finodelk_cbk] 0-s1-client-11: remote operation failed: No such file or directory [2013-10-24 17:06:11.437598] W [client-rpc-fops.c:1579:client3_3_finodelk_cbk] 0-s1-client-11: remote operation failed: No such file or directory [2013-10-24 17:06:11.438225] W [client-rpc-fops.c:1579:client3_3_finodelk_cbk] 0-s1-client-11: remote operation failed: No such file or directory [2013-10-24 17:06:11.438360] W [client-rpc-fops.c:1579:client3_3_finodelk_cbk] 0-s1-client-11: remote operation failed: No such file or directory [2013-10-24 17:06:11.440340] W [client-rpc-fops.c:1579:client3_3_finodelk_cbk] 0-s1-client-11: remote operation failed: No such attached the sosreports