A user has reported an instance of $subject. http://www.gluster.org/pipermail/gluster-users/2016-July/027355.html We have long known this could be a problem, and the long-term solution is a proper QoS implementation. In the interim, we should at least implement one of the measures described in a followup: > We might be able to tweak io-threads (which already runs on the > bricks and already has a global queue) to schedule requests in a > fairer way across clients. Right now it executes them in the > same order that they were read from the network. That tends to > be a bit "unfair" and that should be fixed in the network code, > but that's a much harder task.
REVIEW: http://review.gluster.org/14989 (io-threads: remove least-rate-limit option and code) posted (#4) for review on master by Jeff Darcy (jdarcy)
REVIEW: http://review.gluster.org/14904 (io-threads: distribute work fairly among clients) posted (#6) for review on master by Jeff Darcy (jdarcy)
*** Bug 1360689 has been marked as a duplicate of this bug. ***
COMMIT: http://review.gluster.org/14989 committed in master by Jeff Darcy (jdarcy) ------ commit 25d1bdc094a8408afc074817ec6d63c812c571f9 Author: Jeff Darcy <jdarcy> Date: Thu Jul 21 14:49:55 2016 -0400 io-threads: remove least-rate-limit option and code This will be unnecessary, and mostly in the way, as real fairness guarantees are implemented. Change-Id: Ic61ec1c9e9add58385f1a4eafcfe2cc554ceefc8 BUG: 1360402 Signed-off-by: Jeff Darcy <jdarcy> Reviewed-on: http://review.gluster.org/14989 Smoke: Gluster Build System <jenkins.org> Tested-by: Gluster Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org> NetBSD-regression: Gluster Build System <jenkins.org>
REVIEW: http://review.gluster.org/14904 (io-threads: distribute work fairly among clients) posted (#7) for review on master by Jeff Darcy (jdarcy)
COMMIT: http://review.gluster.org/14904 committed in master by Jeff Darcy (jdarcy) ------ commit 1faf452818fe51fae1f07d17afab910a5e0da0cd Author: Jeff Darcy <jdarcy> Date: Fri Jul 22 13:11:15 2016 -0400 io-threads: distribute work fairly among clients This is the full "queue of queues" approach where each client gets its own queue (per priority) and we round-robin among them. Change-Id: I73955d1b9bb93f2ff781b48dfe2509009c519ec6 BUG: 1360402 Signed-off-by: Jeff Darcy <jdarcy> Reviewed-on: http://review.gluster.org/14904 NetBSD-regression: NetBSD Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org> Smoke: Gluster Build System <jenkins.org>
This has been in for a while, long enough to be part of 3.10, so it's done.