+++ This bug was initially created as a clone of Bug #1457985 +++ +++ This bug was initially created as a clone of Bug #1454602 +++ Description of problem: ======================= On a cifs mount having a dataset of empty directories+ directories with files, started removing few bricks. When issued remove-brick status command, rebalance estimate time shows negative values. I have issued status for almost 21 times during remove-brick rebalance and every time it showed negative values. At the 22nd attempt, the rebalance estimate time showed positive values (at the point, rebalance ran for almost 24 mins) [root@server1 samba]# gluster v remove-brick distrep server1:/bricks/brick6/b6 server2:/bricks/brick6/b6 server3:/bricks/brick6/b6 server4:/bricks/brick6/b6 status Node Rebalanced-files size scanned failures skipped status run time in h:m:s --------- ----------- ----------- ----------- ----------- ----------- ------------ -------------- localhost 2 9.5KB 6 0 0 completed 0:15:16 server1.redhat.com 0 0Bytes 0 0 0 in progress 0:21:32 server2.redhat.com 0 0Bytes 0 0 0 in progress 0:00:00 server3.redhat.com 0 0Bytes 0 0 0 in progress 0:21:21 Estimated time left for rebalance to complete : 2023406814:-21:-32 Version-Release number of selected component (if applicable): 3.8.4-25.el7rhgs.x86_64 How reproducible: ================= 1/1 Steps to Reproduce: =================== 1) Create a distributed-replicate volume and start it. 2) cifs mount the volume on a client. 3) Create a data set of empty directories+ directories with files. 4) Remove few bricks. 5) Keep running remove-brick status command and check "Estimated time left for rebalance to complete " output. Actual results: =============== Rebalance estimate time sometimes shows negative values. Expected results: ================= Rebalance estimate time should not show negative values. From distrep-rebalance.log in sosreport-sysreg-prod.negativevalues-20170523062944: [2017-05-23 05:54:01.319951] I [dht-rebalance.c:4425:gf_defrag_status_get] 0-glusterfs: TIME: num_files_lookedup=0,elapsed time = 51.000000,rate_lookedup=0.000000 [2017-05-23 05:54:01.320001] I [dht-rebalance.c:4428:gf_defrag_status_get] 0-glusterfs: TIME: Estimated total time to complete = 0 seconds [2017-05-23 05:54:01.320012] I [dht-rebalance.c:4431:gf_defrag_status_get] 0-glusterfs: TIME: Seconds left = 18446744073709551565 seconds This skews the results causing the weird result seen. Easily reproducible by running rebalance on a volume with only dirs (no files). --- Additional comment from Worker Ant on 2017-06-01 13:00:56 EDT --- REVIEW: https://review.gluster.org/17448 (cluster/dht: Include dirs in rebalance estimates) posted (#1) for review on master by N Balachandran (nbalacha) --- Additional comment from Worker Ant on 2017-06-07 00:02:27 EDT --- COMMIT: https://review.gluster.org/17448 committed in master by Raghavendra G (rgowdapp) ------ commit c9860430a77f20ddfec532819542bb1d0187c06e Author: N Balachandran <nbalacha> Date: Thu Jun 1 22:13:41 2017 +0530 cluster/dht: Include dirs in rebalance estimates Empty directories were not being considered while calculating rebalance estimates leading to negative time-left values being displayed as part of the rebalance status. Change-Id: I48d41d702e72db30af10e6b87b628baa605afa98 BUG: 1457985 Signed-off-by: N Balachandran <nbalacha> Reviewed-on: https://review.gluster.org/17448 Smoke: Gluster Build System <jenkins.org> NetBSD-regression: NetBSD Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org> Reviewed-by: Amar Tumballi <amarts> Reviewed-by: Raghavendra G <rgowdapp>
REVIEW: https://review.gluster.org/17530 (cluster/dht: Include dirs in rebalance estimates) posted (#1) for review on release-3.10 by N Balachandran (nbalacha)
COMMIT: https://review.gluster.org/17530 committed in release-3.10 by Raghavendra Talur (rtalur) ------ commit f05dd5d6d9b8953bf97c70148bae8eadc68b5c24 Author: N Balachandran <nbalacha> Date: Thu Jun 1 22:13:41 2017 +0530 cluster/dht: Include dirs in rebalance estimates Empty directories were not being considered while calculating rebalance estimates leading to negative time-left values being displayed as part of the rebalance status. > BUG: 1457985 > Signed-off-by: N Balachandran <nbalacha> > Reviewed-on: https://review.gluster.org/17448 > Smoke: Gluster Build System <jenkins.org> > NetBSD-regression: NetBSD Build System <jenkins.org> > CentOS-regression: Gluster Build System <jenkins.org> > Reviewed-by: Amar Tumballi <amarts> > Reviewed-by: Raghavendra G <rgowdapp> Change-Id: I48d41d702e72db30af10e6b87b628baa605afa98 BUG: 1460914 Signed-off-by: N Balachandran <nbalacha> Reviewed-on: https://review.gluster.org/17530 Smoke: Gluster Build System <jenkins.org> NetBSD-regression: NetBSD Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org> Reviewed-by: Raghavendra Talur <rtalur>
REVIEW: https://review.gluster.org/17599 (cluster/dht: Additional checks for rebalance estimates) posted (#1) for review on release-3.10 by N Balachandran (nbalacha)
COMMIT: https://review.gluster.org/17599 committed in release-3.10 by Raghavendra Talur (rtalur) ------ commit b8cdcca098728c5c12cc37606c8837a9b7a13c2f Author: N Balachandran <nbalacha> Date: Mon Jun 19 11:50:28 2017 +0530 cluster/dht: Additional checks for rebalance estimates The rebalance estimates calculation was not handling calculations correctly when no files had been processed, i.e., when rate_lookedup was 0. Now, the estimated time is set to 0 in such scenarios as there is no way for rebalance to figure out how long the process will take to complete without knowing the rate at which the files are being processed. > BUG: 1457985 > Signed-off-by: N Balachandran <nbalacha> > Reviewed-on: https://review.gluster.org/17564 > Smoke: Gluster Build System <jenkins.org> > NetBSD-regression: NetBSD Build System <jenkins.org> > CentOS-regression: Gluster Build System <jenkins.org> > Reviewed-by: Amar Tumballi <amarts> > Reviewed-by: Raghavendra G <rgowdapp> Change-Id: I7b6378e297e1ba139852bcb2239adf2477336b5b BUG: 1460914 Signed-off-by: N Balachandran <nbalacha> Reviewed-on: https://review.gluster.org/17599 Smoke: Gluster Build System <jenkins.org> Reviewed-by: Raghavendra G <rgowdapp> Reviewed-by: Raghavendra Talur <rtalur> NetBSD-regression: NetBSD Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org>
This bug reported is against a version of Gluster that is no longer maintained (or has been EOL'd). See https://www.gluster.org/release-schedule/ for the versions currently maintained. As a result this bug is being closed. If the bug persists on a maintained version of gluster or against the mainline gluster repository, request that it be reopened and the Version field be marked appropriately.