Bug 1167789
| Summary: | DHT: Rebalance- Misleading log messages from __dht_check_free_space function | |||
|---|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Gluster Storage | Reporter: | shylesh <shmohan> | |
| Component: | distribute | Assignee: | Nithya Balachandran <nbalacha> | |
| Status: | CLOSED ERRATA | QA Contact: | Prasad Desala <tdesala> | |
| Severity: | medium | Docs Contact: | ||
| Priority: | low | |||
| Version: | rhgs-3.0 | CC: | amukherj, kramdoss, nbalacha, nchilaka, rgowdapp, rhinduja, rhs-bugs, sheggodu, smohan | |
| Target Milestone: | --- | Keywords: | Reopened, ZStream | |
| Target Release: | RHGS 3.4.0 | |||
| Hardware: | x86_64 | |||
| OS: | Linux | |||
| Whiteboard: | triaged, dht-rebalance-file, dht-log, rebase | |||
| Fixed In Version: | glusterfs-3.12.2-1 | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 1341948 (view as bug list) | Environment: | ||
| Last Closed: | 2018-09-04 06:26:56 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1341948, 1503134 | |||
What is the size of the brick? Also what do you mean by space discrepancy? Should the brick added later be lesser in size compared to the existing bricks? *** This bug has been marked as a duplicate of bug 1341948 *** This is a downstream BZ whereas 1341948 is an upstream one. They cannot be marked Duplicates. Reopening. Verified this BZ on glusterfs version: 3.12.2-4.el7rhgs.x86_64.
Now, we are seeing correct log message when the file is skipped.
[2018-02-20 05:35:11.063121] W [MSGID: 109023] [dht-rebalance.c:962:__dht_check_free_space] 0-distrepx3-dht: data movement of file {blocks:1 name:(/1/linux-4.6.4/Documentation/devicetree/bindings/display/panel/innolux,zj070na-01p.txt) } would result in dst node (distrepx3-replicate-1:41275176) having lower disk space then the source node (distrepx3-replicate-0:41449024).Skipping file.
[2018-02-20 05:35:11.400035] W [MSGID: 109023] [dht-rebalance.c:962:__dht_check_free_space] 0-distrepx3-dht: data movement of file {blocks:1 name:(/1/linux-4.6.4/Documentation/devicetree/bindings/display/panel/lg,lb070wv8.txt) } would result in dst node (distrepx3-replicate-3:41444376) having lower disk space then the source node (distrepx3-replicate-2:41445288).Skipping file.
[2018-02-20 05:35:12.421654] W [MSGID: 109023] [dht-rebalance.c:962:__dht_check_free_space] 0-distrepx3-dht: data movement of file {blocks:6 name:(/1/linux-4.6.4/Documentation/devicetree/bindings/display/ti/ti,omap4-dss.txt) } would result in dst node (distrepx3-replicate-1:41275184) having lower disk space then the source node (distrepx3-replicate-0:41449032).Skipping file.
[2018-02-20 05:35:14.544070] W [MSGID: 109023] [dht-rebalance.c:962:__dht_check_free_space] 0-distrepx3-dht: data movement of file {blocks:3 name:(/1/linux-4.6.4/Documentation/devicetree/bindings/dma/sun4i-dma.txt) } would result in dst node (distrepx3-replicate-1:41275384) having lower disk space then the source node (distrepx3-replicate-0:41449112).Skipping file.
Here available space on source is distrepx3-replicate-0:41449112 and on destination is distrepx3-replicate-1:41275384, which is correct and expected to skip the file migration.
Hence, moving this BZ to Verified.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2018:2607 |
Description of problem: misleading message from logs about available disk space Version-Release number of selected component (if applicable): 3.6.0.33-1.el6rhs.x86_64 How reproducible: always Steps to Reproduce: 1. create a volume and data 2. add one more brick , make sure that there is some space discrepancy 3. start the rebalance without force option gluster volume rebalance <vol> start 4. check the status of rebalance and make sure that some skipped files are there Actual results: 2014-11-25 09:10:32.433141] I [dht-rebalance.c:902:dht_migrate_file] 0-gs-dht: /file9: attempting to move from gs-client-0 to gs-client-4 [2014-11-25 09:10:32.437164] W [MSGID: 109023] [dht-rebalance.c:568:__dht_check_free_space] 0-gs-dht: data movement attempted from node (gs-client-0:209540896) with higher disk space to a node (gs-client-4:209540960) with lesser disk space, file { blocks:2048, name:(/file9) } [2014-11-25 09:10:32.438441] I [dht-common.c:1563:dht_lookup_everywhere_cbk] 0-gs-dht: attempting deletion of stale linkfile /file10 on gs-client-4 (hashed subvol is gs-client-6) Here available space on source is gs-client-0:209540896 and on destination is gs-client-4:209540960 but log says source has higher space and destination has lesser which is misleading . since calculation of free space is done in the following way if ((dst_statfs_blocks - stbuf->ia_blocks) < (src_statfs_blocks + stbuf->ia_blocks)) the proper values has to be printed in the logs