Bug 1167789 - DHT: Rebalance- Misleading log messages from __dht_check_free_space function
Summary: DHT: Rebalance- Misleading log messages from __dht_check_free_space function
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: distribute
Version: rhgs-3.0
Hardware: x86_64
OS: Linux
low
medium
Target Milestone: ---
: RHGS 3.4.0
Assignee: Nithya Balachandran
QA Contact: Prasad Desala
URL:
Whiteboard: triaged, dht-rebalance-file, dht-log,...
Depends On:
Blocks: 1341948 1503134
TreeView+ depends on / blocked
 
Reported: 2014-11-25 12:33 UTC by shylesh
Modified: 2018-09-04 06:28 UTC (History)
9 users (show)

Fixed In Version: glusterfs-3.12.2-1
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1341948 (view as bug list)
Environment:
Last Closed: 2018-09-04 06:26:56 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2018:2607 0 None None None 2018-09-04 06:28:39 UTC

Description shylesh 2014-11-25 12:33:13 UTC
Description of problem:
misleading message from logs about available disk space

Version-Release number of selected component (if applicable):
3.6.0.33-1.el6rhs.x86_64

How reproducible:
always

Steps to Reproduce:
1. create a volume and data 
2. add one more brick , make sure that there is some space discrepancy 
3. start the rebalance without force option
gluster volume rebalance <vol> start 
4. check the status of rebalance and make sure that some skipped files are there

Actual results:
2014-11-25 09:10:32.433141] I [dht-rebalance.c:902:dht_migrate_file] 0-gs-dht: /file9: attempting to move from gs-client-0 to gs-client-4
[2014-11-25 09:10:32.437164] W [MSGID: 109023] [dht-rebalance.c:568:__dht_check_free_space] 0-gs-dht: data movement attempted from node (gs-client-0:209540896) with higher disk space to a node (gs-client-4:209540960) with lesser disk space, file { blocks:2048, name:(/file9) }


[2014-11-25 09:10:32.438441] I [dht-common.c:1563:dht_lookup_everywhere_cbk] 0-gs-dht: attempting deletion of stale linkfile /file10 on gs-client-4 (hashed subvol is gs-client-6)



Here available space on source is gs-client-0:209540896 and on destination is gs-client-4:209540960 but log says source has higher space and destination has lesser which is misleading .

since calculation of free space is done in the following way

if ((dst_statfs_blocks - stbuf->ia_blocks) <
    (src_statfs_blocks + stbuf->ia_blocks)) 


the proper values has to be printed in the logs

Comment 2 Sakshi 2015-04-22 13:03:44 UTC
What is the size of the brick? Also what do you mean by space discrepancy? Should the brick added later be lesser in size compared to the existing bricks?

Comment 3 ankit 2016-11-02 05:42:35 UTC

*** This bug has been marked as a duplicate of bug 1341948 ***

Comment 4 Nithya Balachandran 2016-11-02 06:28:41 UTC
This is a downstream BZ whereas 1341948 is an upstream one. They cannot be marked Duplicates.

Reopening.

Comment 10 Prasad Desala 2018-02-20 07:22:27 UTC
Verified this BZ on glusterfs version: 3.12.2-4.el7rhgs.x86_64.

Now, we are seeing correct log message when the file is skipped.

[2018-02-20 05:35:11.063121] W [MSGID: 109023] [dht-rebalance.c:962:__dht_check_free_space] 0-distrepx3-dht: data movement of file {blocks:1 name:(/1/linux-4.6.4/Documentation/devicetree/bindings/display/panel/innolux,zj070na-01p.txt) } would result in dst node (distrepx3-replicate-1:41275176) having lower disk space then the source node (distrepx3-replicate-0:41449024).Skipping file.
[2018-02-20 05:35:11.400035] W [MSGID: 109023] [dht-rebalance.c:962:__dht_check_free_space] 0-distrepx3-dht: data movement of file {blocks:1 name:(/1/linux-4.6.4/Documentation/devicetree/bindings/display/panel/lg,lb070wv8.txt) } would result in dst node (distrepx3-replicate-3:41444376) having lower disk space then the source node (distrepx3-replicate-2:41445288).Skipping file.
[2018-02-20 05:35:12.421654] W [MSGID: 109023] [dht-rebalance.c:962:__dht_check_free_space] 0-distrepx3-dht: data movement of file {blocks:6 name:(/1/linux-4.6.4/Documentation/devicetree/bindings/display/ti/ti,omap4-dss.txt) } would result in dst node (distrepx3-replicate-1:41275184) having lower disk space then the source node (distrepx3-replicate-0:41449032).Skipping file.

[2018-02-20 05:35:14.544070] W [MSGID: 109023] [dht-rebalance.c:962:__dht_check_free_space] 0-distrepx3-dht: data movement of file {blocks:3 name:(/1/linux-4.6.4/Documentation/devicetree/bindings/dma/sun4i-dma.txt) } would result in dst node (distrepx3-replicate-1:41275384) having lower disk space then the source node (distrepx3-replicate-0:41449112).Skipping file.

Here available space on source is distrepx3-replicate-0:41449112 and on destination is distrepx3-replicate-1:41275384, which is correct and expected to skip the file migration.

Hence, moving this BZ to Verified.

Comment 12 errata-xmlrpc 2018-09-04 06:26:56 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2607


Note You need to log in before you can comment on or make changes to this bug.