Problem statement: ============================ Rebalance is not considering the brick sizes while fixing the layout of the volume Steps/procedure: 1. create a distribute volume using one brick of 100GB . 2. Mount it on the client using FUSE and create directory and 1000 files 3. add brick of 200GB from the another node and run the rebalance from the same node Actual results: ================ Though Brick2 is of 200GB, it is holding only 327 and another brick has 676. Direcotry ranges are given below [root@rhs-client9 dht4]# getfattr -d -m . -e hex /rhs/brick2/dht4/data getfattr: Removing leading '/' from absolute path names # file: rhs/brick2/dht4/data security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000 trusted.gfid=0x2bcf9f94144a4decb533a419885784cc trusted.glusterfs.dht=0x0000000100000000aaa972d0ffffffff (200 GB Brick) [root@rhs-client4 dht4]# getfattr -d -m . -e hex /rhs/brick1/dht4/data getfattr: Removing leading '/' from absolute path names # file: rhs/brick1/dht4/data security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000 trusted.gfid=0x2bcf9f94144a4decb533a419885784cc trusted.glusterfs.dht=0x000000010000000000000000aaa972cf (100 GB Brick) Expected results: ================== while fixing the layout re-balance should consider the brick sizes Output: =================== [root@rhs-client4 dht4]# gluster vol status dht4 Status of volume: dht4 Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick rhs-client4.lab.eng.blr.redhat.com:/r hs/brick1/dht4 49158 0 Y 20117 Brick rhs-client9.lab.eng.blr.redhat.com:/r hs/brick2/dht4 49157 0 Y 29628 NFS Server on localhost 2049 0 Y 20301 NFS Server on rhs-client39.lab.eng.blr.redh at.com N/A N/A N N/A NFS Server on rhs-client9.lab.eng.blr.redha t.com N/A N/A N N/A Task Status of Volume dht4 ------------------------------------------------------------------------------ Task : Rebalance ID : b93f08b3-e59c-4e30-bd0f-b405e553bdb3 Status : completed [root@rhs-client9 dht4]# df -h | grep brick2 /dev/mapper/rhel_rhs--client9-vol1 200G 60M 200G 1% /rhs/brick2 [root@rhs-client4 dht4]# df -h | grep brick1 /dev/mapper/rhgs_rhs--client4-vol1 100G 84M 100G 1% /rhs/brick1
The ranges allocated are: >>> 0xffffffff - 0xaaa972d0 1431735599 >>> 0xaaa972cf 2863231695 Though the ranges are in the ratio 1:2, they are allocated to wrong bricks. Large range is allocated to smaller brick. Need to fix it.
http://review.gluster.org/15403 posted upstream for review.
Upstream mainline : http://review.gluster.org/15403 Upstream 3.8 : http://review.gluster.org/15422 downstream patch : https://code.engineering.redhat.com/gerrit/#/c/84881/
Verified this BZ with glusterfs version: 3.8.4-3.el7rhgs.x86_64. Here are the steps that were performed, 1) Created a distribute volume using one brick of 100GB . 2) Mounted it on the client using FUSE and created directory and 1000 files. 3) added brick of 200GB from the another node and ran the rebalance from the same node. Rebalance is considering the brick sizes while setting the layout, where large range is allocated to a large brick and a smaller range to a small brick. Hence, moving this BZ to Verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2017-0486.html