Previously, the quota limit configuration stored in the extended attributes of the directory would not be present in the newly added brick. The quota list command may fail to list the limits on the directory when only the newly added bricks are available. Now, with this update, the configuration is copied on the newly added bricks and the quota list command does not fail.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
http://rhn.redhat.com/errata/RHBA-2013-1769.html
Description of problem: After doing add-brick and fix-layout on a distributed-replicate volume which has already reached the quota limit, file which gets hashed to the newly added bricks will violate quota Version-Release number of selected component (if applicable): [root@gqac023 mnt]# rpm -qa| grep gluster glusterfs-fuse-3.4.0.20rhsquota5-1.el6rhs.x86_64 glusterfs-server-3.4.0.20rhsquota5-1.el6rhs.x86_64 glusterfs-3.4.0.20rhsquota5-1.el6rhs.x86_64 glusterfs-geo-replication-3.4.0.20rhsquota5-1.el6rhs.x86_64 glusterfs-api-devel-3.4.0.20rhsquota5-1.el6rhs.x86_64 glusterfs-rdma-3.4.0.20rhsquota5-1.el6rhs.x86_64 glusterfs-debuginfo-3.4.0.20rhsquota5-1.el6rhs.x86_64 glusterfs-libs-3.4.0.20rhsquota5-1.el6rhs.x86_64 glusterfs-api-3.4.0.20rhsquota5-1.el6rhs.x86_64 glusterfs-devel-3.4.0.20rhsquota5-1.el6rhs.x86_64 How reproducible: always Steps to Reproduce: 1.created a distributed-replica volume of 3x2 configuration and set the hard quota limit to 1GB on / 2.Mounted the volume and created a directory and set the quota to 1GB and inside this directory created a file of size 1GB 3.now further file creation fails with "quota exceeded" message 4. added one more pair of brick and ran "rebalance fix-layout" 5. Now tried to create more files Actual results: The files which gets hashed to the newly added pair will not respect quota as they get crated and violating quota Expected results: Additional info: Volume Name: replica Type: Distributed-Replicate Volume ID: fba43d74-bbd0-407e-beda-67ebeb4fb9c9 Status: Started Number of Bricks: 4 x 2 = 8 Transport-type: tcp Bricks: Brick1: gqac022.sbu.lab.eng.bos.redhat.com:/home/r0 Brick2: gqac023.sbu.lab.eng.bos.redhat.com:/home/r0 Brick3: gqac024.sbu.lab.eng.bos.redhat.com:/home/r1 Brick4: gqac025.sbu.lab.eng.bos.redhat.com:/home/r1 Brick5: gqac022.sbu.lab.eng.bos.redhat.com:/home/r2 Brick6: gqac023.sbu.lab.eng.bos.redhat.com:/home/r2 Brick7: gqac024.sbu.lab.eng.bos.redhat.com:/home/r3 ===>newly added pair Brick8: gqac025.sbu.lab.eng.bos.redhat.com:/home/r3 ===>newly added pair Options Reconfigured: features.quota: on [root@gqac023 mnt]# gluster v quota replica list Path Hard-limit Soft-limit Used Available -------------------------------------------------------------------------------- / 1.0GB 8977070246986897596 3.8GB 0Bytes /test 1.0GB 8977070246986897596 3.8GB 0Bytes Cluster info =========== RHS nodes --------- gqac022.sbu.lab.eng.bos.redhat.com gqac023.sbu.lab.eng.bos.redhat.com gqac024.sbu.lab.eng.bos.redhat.com gqac025.sbu.lab.eng.bos.redhat.com Mounted on ========== gqac022.sbu.lab.eng.bos.redhat.com Mount point ============ /mnt1 xattrs from the bricks for / ============================ pair1 ===== [root@gqac022 test]# getfattr -d -m . -e hex /home/r0/ getfattr: Removing leading '/' from absolute path names # file: home/r0/ security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a66696c655f743a733000 trusted.gfid=0x00000000000000000000000000000001 trusted.glusterfs.dht=0x0000000100000000000000003ffffffe trusted.glusterfs.quota.dirty=0x3000 trusted.glusterfs.quota.limit-set=0x00000000400000007c94f5d3bb61d8bc trusted.glusterfs.quota.size=0x00000000046f0000 trusted.glusterfs.volume-id=0xfba43d74bbd0407ebeda67ebeb4fb9c9 [root@gqac023 mnt]# getfattr -d -m . -e hex /home/r0 getfattr: Removing leading '/' from absolute path names # file: home/r0 security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a66696c655f743a733000 trusted.gfid=0x00000000000000000000000000000001 trusted.glusterfs.dht=0x0000000100000000000000003ffffffe trusted.glusterfs.quota.dirty=0x3000 trusted.glusterfs.quota.limit-set=0x00000000400000007c94f5d3bb61d8bc trusted.glusterfs.quota.size=0x00000000046f0000 trusted.glusterfs.volume-id=0xfba43d74bbd0407ebeda67ebeb4fb9c9 pair 2 ====== [root@gqac024 in]# getfattr -d -m . -e hex /home/r1 getfattr: Removing leading '/' from absolute path names # file: home/r1 security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a66696c655f743a733000 trusted.gfid=0x00000000000000000000000000000001 trusted.glusterfs.dht=0x00000001000000003fffffff7ffffffd trusted.glusterfs.quota.dirty=0x3000 trusted.glusterfs.quota.limit-set=0x00000000400000007c94f5d3bb61d8bc trusted.glusterfs.quota.size=0x000000003b9aca00 trusted.glusterfs.volume-id=0xfba43d74bbd0407ebeda67ebeb4fb9c9 [root@gqac025 home]# getfattr -d -m . -e hex /home/r1 getfattr: Removing leading '/' from absolute path names # file: home/r1 security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a66696c655f743a733000 trusted.gfid=0x00000000000000000000000000000001 trusted.glusterfs.dht=0x00000001000000003fffffff7ffffffd trusted.glusterfs.quota.dirty=0x3000 trusted.glusterfs.quota.limit-set=0x00000000400000007c94f5d3bb61d8bc trusted.glusterfs.quota.size=0x000000003b9aca00 trusted.glusterfs.volume-id=0xfba43d74bbd0407ebeda67ebeb4fb9c9 pair3 ===== [root@gqac022 test]# getfattr -d -m . -e hex /home/r2 getfattr: Removing leading '/' from absolute path names # file: home/r2 security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a66696c655f743a733000 trusted.gfid=0x00000000000000000000000000000001 trusted.glusterfs.dht=0x00000001000000007ffffffebffffffc trusted.glusterfs.quota.dirty=0x3000 trusted.glusterfs.volume-id=0xfba43d74bbd0407ebeda67ebeb4fb9c9 [root@gqac023 mnt]# getfattr -d -m . -e hex /home/r2 getfattr: Removing leading '/' from absolute path names # file: home/r2 security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a66696c655f743a733000 trusted.gfid=0x00000000000000000000000000000001 trusted.glusterfs.dht=0x00000001000000007ffffffebffffffc trusted.glusterfs.quota.dirty=0x3000 trusted.glusterfs.volume-id=0xfba43d74bbd0407ebeda67ebeb4fb9c9 pair 4 ====== [root@gqac024 in]# getfattr -d -m . -e hex /home/r3 getfattr: Removing leading '/' from absolute path names # file: home/r3 security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a66696c655f743a733000 trusted.gfid=0x00000000000000000000000000000001 trusted.glusterfs.dht=0x0000000100000000bffffffdffffffff trusted.glusterfs.quota.dirty=0x3000 trusted.glusterfs.quota.size=0x00000000b2d05e00 trusted.glusterfs.volume-id=0xfba43d74bbd0407ebeda67ebeb4fb9c9 [root@gqac025 home]# getfattr -d -m . -e hex /home/r3 getfattr: Removing leading '/' from absolute path names # file: home/r3 security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a66696c655f743a733000 trusted.gfid=0x00000000000000000000000000000001 trusted.glusterfs.dht=0x0000000100000000bffffffdffffffff trusted.glusterfs.quota.dirty=0x3000 trusted.glusterfs.quota.size=0x00000000b2d05e00 trusted.glusterfs.volume-id=0xfba43d74bbd0407ebeda67ebeb4fb9c9 xattrs dump for /test ===================== pair1 ===== [root@gqac022 test]# getfattr -d -m . -e hex /home/r0/test getfattr: Removing leading '/' from absolute path names # file: home/r0/test security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a66696c655f743a733000 trusted.gfid=0x86f12f86ab6a4700be9d894faf33a450 trusted.glusterfs.dht=0x0000000100000000000000003ffffffe trusted.glusterfs.quota.00000000-0000-0000-0000-000000000001.contri=0x00000000046f0000 trusted.glusterfs.quota.dirty=0x3000 trusted.glusterfs.quota.limit-set=0x00000000400000007c94f5d3bb61d8bc trusted.glusterfs.quota.size=0x00000000046f0000 [root@gqac023 mnt]# getfattr -d -m . -e hex /home/r0/test getfattr: Removing leading '/' from absolute path names # file: home/r0/test security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a66696c655f743a733000 trusted.gfid=0x86f12f86ab6a4700be9d894faf33a450 trusted.glusterfs.dht=0x0000000100000000000000003ffffffe trusted.glusterfs.quota.00000000-0000-0000-0000-000000000001.contri=0x00000000046f0000 trusted.glusterfs.quota.dirty=0x3000 trusted.glusterfs.quota.limit-set=0x00000000400000007c94f5d3bb61d8bc trusted.glusterfs.quota.size=0x00000000046f0000 pair2 ===== [root@gqac024 in]# getfattr -d -m . -e hex /home/r1/test getfattr: Removing leading '/' from absolute path names # file: home/r1/test security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a66696c655f743a733000 trusted.gfid=0x86f12f86ab6a4700be9d894faf33a450 trusted.glusterfs.dht=0x00000001000000003fffffff7ffffffd trusted.glusterfs.quota.00000000-0000-0000-0000-000000000001.contri=0x000000003b9aca00 trusted.glusterfs.quota.dirty=0x3000 trusted.glusterfs.quota.limit-set=0x00000000400000007c94f5d3bb61d8bc trusted.glusterfs.quota.size=0x000000003b9aca00 [root@gqac025 home]# getfattr -d -m . -e hex /home/r1/test getfattr: Removing leading '/' from absolute path names # file: home/r1/test security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a66696c655f743a733000 trusted.gfid=0x86f12f86ab6a4700be9d894faf33a450 trusted.glusterfs.dht=0x00000001000000003fffffff7ffffffd trusted.glusterfs.quota.00000000-0000-0000-0000-000000000001.contri=0x000000003b9aca00 trusted.glusterfs.quota.dirty=0x3000 trusted.glusterfs.quota.limit-set=0x00000000400000007c94f5d3bb61d8bc trusted.glusterfs.quota.size=0x000000003b9aca00 pair3 ======= [root@gqac022 test]# getfattr -d -m . -e hex /home/r2/test getfattr: Removing leading '/' from absolute path names # file: home/r2/test security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a66696c655f743a733000 trusted.gfid=0x86f12f86ab6a4700be9d894faf33a450 trusted.glusterfs.dht=0x0000000100000000bffffffdffffffff trusted.glusterfs.quota.00000000-0000-0000-0000-000000000001.contri=0x0000000000000000 trusted.glusterfs.quota.dirty=0x3000 trusted.glusterfs.quota.limit-set=0x00000000400000007c94f5d3bb61d8bc trusted.glusterfs.quota.size=0x0000000000000000 [root@gqac023 mnt]# getfattr -d -m . -e hex /home/r2/test getfattr: Removing leading '/' from absolute path names # file: home/r2/test security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a66696c655f743a733000 trusted.gfid=0x86f12f86ab6a4700be9d894faf33a450 trusted.glusterfs.dht=0x0000000100000000bffffffdffffffff trusted.glusterfs.quota.00000000-0000-0000-0000-000000000001.contri=0x0000000000000000 trusted.glusterfs.quota.dirty=0x3000 trusted.glusterfs.quota.limit-set=0x00000000400000007c94f5d3bb61d8bc trusted.glusterfs.quota.size=0x0000000000000000 pair4 ======= [root@gqac024 in]# getfattr -d -m . -e hex /home/r3/test getfattr: Removing leading '/' from absolute path names # file: home/r3/test security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a66696c655f743a733000 trusted.gfid=0x86f12f86ab6a4700be9d894faf33a450 trusted.glusterfs.dht=0x00000001000000007ffffffebffffffc trusted.glusterfs.quota.00000000-0000-0000-0000-000000000001.contri=0x00000000b2d05e00 trusted.glusterfs.quota.dirty=0x3000 trusted.glusterfs.quota.size=0x00000000b2d05e00 [root@gqac025 home]# getfattr -d -m . -e hex /home/r3/test getfattr: Removing leading '/' from absolute path names # file: home/r3/test security.selinux=0x756e636f6e66696e65645f753a6f626a6563745f723a66696c655f743a733000 trusted.gfid=0x86f12f86ab6a4700be9d894faf33a450 trusted.glusterfs.dht=0x00000001000000007ffffffebffffffc trusted.glusterfs.quota.00000000-0000-0000-0000-000000000001.contri=0x00000000b2d05e00 trusted.glusterfs.quota.dirty=0x3000 trusted.glusterfs.quota.size=0x00000000b2d05e00 ======================================================================== the files which are hashed to pair4 are violatoing quota [root@gqac022 test]# dd if=/dev/urandom of=file23 bs=1GB count=1 dd: opening `file23': Disk quota exceeded [root@gqac022 test]# dd if=/dev/urandom of=file24 bs=1GB count=1 dd: opening `file24': Disk quota exceeded [root@gqac022 test]# dd if=/dev/urandom of=file25 bs=1GB count=1 dd: opening `file25': Disk quota exceeded [root@gqac022 test]# dd if=/dev/urandom of=file26 bs=1GB count=1 1+0 records in 1+0 records out 1000000000 bytes (1.0 GB) copied, 130.736 s, 7.6 MB/s You have new mail in /var/spool/mail/root [root@gqac022 test]# dd if=/dev/urandom of=file27 bs=1GB count=1 1+0 records in 1+0 records out 1000000000 bytes (1.0 GB) copied, 130.923 s, 7.6 MB/s [root@gqac022 test]# dd if=/dev/urandom of=file28 bs=1GB count=1 dd: opening `file28': Disk quota exceeded You have new mail in /var/spool/mail/root [root@gqac022 test]# dd if=/dev/urandom of=file29 bs=1GB count=1 1+0 records in 1+0 records out 1000000000 bytes (1.0 GB) copied, 131.003 s, 7.6 MB/s attached the sosreports