+++ This bug was initially created as a clone of Bug #1475136 +++ Description of problem: ----------------------- A regression seems to have been introduced in recent bits on large file seq reads : 3.2 : 2480044.05 kB/sec 3.8.4-35 : 1538178.2 kB/sec Regression : ~38% This is on a vanilla volume,without PR,NL,mdcache. Version-Release number of selected component (if applicable): ------------------------------------------------------------- 3.8.4-35 How reproducible: ----------------- 100% Additional info: ---------------- Volume Name: testvol Type: Distributed-Replicate Volume ID: 4b52bfb8-28fd-4e0f-8ee0-eb8116a296c4 Status: Started Snapshot Count: 0 Number of Bricks: 2 x 2 = 4 Transport-type: tcp Bricks: Brick1: server1:/bricks/testvol_brick0 Brick2: server2:/bricks/testvol_brick1 Brick3: server3:/bricks/testvol_brick2 Brick4: server4:/bricks/testvol_brick3 Options Reconfigured: server.allow-insecure: on performance.stat-prefetch: off transport.address-family: inet nfs.disable: on
The changes made in https://review.gluster.org/#/c/17630/ have introduced a perf regression. The code has been changed to check if the fd is open on the dst subvol only if the FOP returns with EBADFD.
REVIEW: https://review.gluster.org/17922 (cluster/dht: Check for open fd only on EBADF) posted (#1) for review on master by N Balachandran (nbalacha)
REVIEW: https://review.gluster.org/17922 (cluster/dht: Check for open fd only on EBADF) posted (#2) for review on master by N Balachandran (nbalacha)
COMMIT: https://review.gluster.org/17922 committed in master by Raghavendra G (rgowdapp) ------ commit 91c9f4a19fde4894576b398252c77f730832a26a Author: N Balachandran <nbalacha> Date: Mon Jul 31 12:32:59 2017 +0530 cluster/dht: Check for open fd only on EBADF DHT fd based fops will now check if the fd is open on the cached subvol only if the call fails with EBADF. This will improve performance for scenarios where a rebalance is not running which would be most of the time. Change-Id: Idfaeb8927af769c6110d07a165a0fe2307369239 BUG: 1476665 Signed-off-by: N Balachandran <nbalacha> Reviewed-on: https://review.gluster.org/17922 Smoke: Gluster Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org> Reviewed-by: Raghavendra G <rgowdapp>
REVIEW: https://review.gluster.org/17970 (Revert "cluster/dht: Check for open fd only on EBADF") posted (#1) for review on master by N Balachandran (nbalacha)
COMMIT: https://review.gluster.org/17970 committed in master by N Balachandran (nbalacha) ------ commit 6d1068ddb35be19df36210c9fcaa7ce97e2a376a Author: N Balachandran <nbalacha> Date: Fri Aug 4 08:52:31 2017 +0530 Revert "cluster/dht: Check for open fd only on EBADF" This reverts commit 91c9f4a19fde4894576b398252c77f730832a26a. This patch needs to be reworked. Change-Id: I4c24f647c2b1abc68fc4e9fe6eb810418e2033aa BUG: 1476665 Signed-off-by: N Balachandran <nbalacha> Reviewed-on: https://review.gluster.org/17970 Smoke: Gluster Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org>
REVIEW: https://review.gluster.org/17976 (cluster/dht: Check for open fd only on EBADF) posted (#1) for review on master by N Balachandran (nbalacha)
REVIEW: https://review.gluster.org/17976 (cluster/dht: Check for open fd only on EBADF) posted (#2) for review on master by N Balachandran (nbalacha)
COMMIT: https://review.gluster.org/17976 committed in master by Raghavendra G (rgowdapp) ------ commit cdca1cb26a0aba390c6d8485c0d6d95e22ffc8bd Author: N Balachandran <nbalacha> Date: Fri Aug 4 14:46:38 2017 +0530 cluster/dht: Check for open fd only on EBADF DHT fd based fops used to check if the fd was open on the cached subvol before winding the call. However, this introduced a performance regression of about 30% for reads. This check was introduced to handle cases where files were migrated while IOs were happening. As this is not the common case, dht will now check if the fd is open on the cached subvol only if the call fails with EBADF. This will prevent a performance hit where a rebalance is not running. Change-Id: I2035a858d63c3fcd22bb634055bbb0ad01686808 BUG: 1476665 Signed-off-by: N Balachandran <nbalacha> Reviewed-on: https://review.gluster.org/17976 Smoke: Gluster Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org> Reviewed-by: Amar Tumballi <amarts> Reviewed-by: Susant Palai <spalai> Reviewed-by: Raghavendra G <rgowdapp>
One more patch required.
REVIEW: https://review.gluster.org/17999 (cluster/dht: EBADF handling for fremovexattr and fsetxattr) posted (#1) for review on master by N Balachandran (nbalacha)
COMMIT: https://review.gluster.org/17999 committed in master by Raghavendra G (rgowdapp) ------ commit 747a08d34e2a1e94d7fce68a3577370288bb1955 Author: N Balachandran <nbalacha> Date: Tue Aug 8 22:33:24 2017 +0530 cluster/dht: EBADF handling for fremovexattr and fsetxattr Add EBADF handling for dht_fremovexattr and dht_fsetxattr. Change-Id: Ide0d5812dae79655d2565157e5baabcd753b4309 BUG: 1476665 Signed-off-by: N Balachandran <nbalacha> Reviewed-on: https://review.gluster.org/17999 Smoke: Gluster Build System <jenkins.org> Reviewed-by: Shyamsundar Ranganathan <srangana> CentOS-regression: Gluster Build System <jenkins.org> Reviewed-by: Raghavendra G <rgowdapp>
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.13.0, please open a new bug report. glusterfs-3.13.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://lists.gluster.org/pipermail/announce/2017-December/000087.html [2] https://www.gluster.org/pipermail/gluster-users/