Red Hat Bugzilla – Bug 1167789
DHT: Rebalance- Misleading log messages from __dht_check_free_space function
Last modified: 2018-09-04 02:28:39 EDT
Description of problem: misleading message from logs about available disk space Version-Release number of selected component (if applicable): 3.6.0.33-1.el6rhs.x86_64 How reproducible: always Steps to Reproduce: 1. create a volume and data 2. add one more brick , make sure that there is some space discrepancy 3. start the rebalance without force option gluster volume rebalance <vol> start 4. check the status of rebalance and make sure that some skipped files are there Actual results: 2014-11-25 09:10:32.433141] I [dht-rebalance.c:902:dht_migrate_file] 0-gs-dht: /file9: attempting to move from gs-client-0 to gs-client-4 [2014-11-25 09:10:32.437164] W [MSGID: 109023] [dht-rebalance.c:568:__dht_check_free_space] 0-gs-dht: data movement attempted from node (gs-client-0:209540896) with higher disk space to a node (gs-client-4:209540960) with lesser disk space, file { blocks:2048, name:(/file9) } [2014-11-25 09:10:32.438441] I [dht-common.c:1563:dht_lookup_everywhere_cbk] 0-gs-dht: attempting deletion of stale linkfile /file10 on gs-client-4 (hashed subvol is gs-client-6) Here available space on source is gs-client-0:209540896 and on destination is gs-client-4:209540960 but log says source has higher space and destination has lesser which is misleading . since calculation of free space is done in the following way if ((dst_statfs_blocks - stbuf->ia_blocks) < (src_statfs_blocks + stbuf->ia_blocks)) the proper values has to be printed in the logs
What is the size of the brick? Also what do you mean by space discrepancy? Should the brick added later be lesser in size compared to the existing bricks?
*** This bug has been marked as a duplicate of bug 1341948 ***
This is a downstream BZ whereas 1341948 is an upstream one. They cannot be marked Duplicates. Reopening.
Verified this BZ on glusterfs version: 3.12.2-4.el7rhgs.x86_64. Now, we are seeing correct log message when the file is skipped. [2018-02-20 05:35:11.063121] W [MSGID: 109023] [dht-rebalance.c:962:__dht_check_free_space] 0-distrepx3-dht: data movement of file {blocks:1 name:(/1/linux-4.6.4/Documentation/devicetree/bindings/display/panel/innolux,zj070na-01p.txt) } would result in dst node (distrepx3-replicate-1:41275176) having lower disk space then the source node (distrepx3-replicate-0:41449024).Skipping file. [2018-02-20 05:35:11.400035] W [MSGID: 109023] [dht-rebalance.c:962:__dht_check_free_space] 0-distrepx3-dht: data movement of file {blocks:1 name:(/1/linux-4.6.4/Documentation/devicetree/bindings/display/panel/lg,lb070wv8.txt) } would result in dst node (distrepx3-replicate-3:41444376) having lower disk space then the source node (distrepx3-replicate-2:41445288).Skipping file. [2018-02-20 05:35:12.421654] W [MSGID: 109023] [dht-rebalance.c:962:__dht_check_free_space] 0-distrepx3-dht: data movement of file {blocks:6 name:(/1/linux-4.6.4/Documentation/devicetree/bindings/display/ti/ti,omap4-dss.txt) } would result in dst node (distrepx3-replicate-1:41275184) having lower disk space then the source node (distrepx3-replicate-0:41449032).Skipping file. [2018-02-20 05:35:14.544070] W [MSGID: 109023] [dht-rebalance.c:962:__dht_check_free_space] 0-distrepx3-dht: data movement of file {blocks:3 name:(/1/linux-4.6.4/Documentation/devicetree/bindings/dma/sun4i-dma.txt) } would result in dst node (distrepx3-replicate-1:41275384) having lower disk space then the source node (distrepx3-replicate-0:41449112).Skipping file. Here available space on source is distrepx3-replicate-0:41449112 and on destination is distrepx3-replicate-1:41275384, which is correct and expected to skip the file migration. Hence, moving this BZ to Verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2018:2607