Bug 1479303

+++ This bug was initially created as a clone of Bug #1476665 +++

+++ This bug was initially created as a clone of Bug #1475136 +++

Description of problem:
-----------------------

A regression seems to have been introduced in recent bits on large file seq reads :

3.2 : 2480044.05 kB/sec	
3.8.4-35 : 1538178.2 kB/sec


Regression : ~38%

This is on a vanilla volume,without PR,NL,mdcache.

Version-Release number of selected component (if applicable):
-------------------------------------------------------------

3.8.4-35

How reproducible:
-----------------

100%

Additional info:
----------------
Volume Name: testvol
Type: Distributed-Replicate
Volume ID: 4b52bfb8-28fd-4e0f-8ee0-eb8116a296c4
Status: Started
Snapshot Count: 0
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Bricks:
Brick1: server1:/bricks/testvol_brick0
Brick2: server2:/bricks/testvol_brick1
Brick3: server3:/bricks/testvol_brick2
Brick4: server4:/bricks/testvol_brick3
Options Reconfigured:
server.allow-insecure: on
performance.stat-prefetch: off
transport.address-family: inet
nfs.disable: on

--- Additional comment from Nithya Balachandran on 2017-07-31 02:58:32 EDT ---

The changes made in https://review.gluster.org/#/c/17630/ have introduced a perf regression. 


The code has been changed to check if the fd is open on the dst subvol only if the FOP returns with EBADFD.

--- Additional comment from Worker Ant on 2017-07-31 03:06:04 EDT ---

REVIEW: https://review.gluster.org/17922 (cluster/dht: Check for open fd only on EBADF) posted (#1) for review on master by N Balachandran (nbalacha)

--- Additional comment from Worker Ant on 2017-07-31 04:10:16 EDT ---

REVIEW: https://review.gluster.org/17922 (cluster/dht: Check for open fd only on EBADF) posted (#2) for review on master by N Balachandran (nbalacha)

--- Additional comment from Worker Ant on 2017-07-31 10:00:25 EDT ---

COMMIT: https://review.gluster.org/17922 committed in master by Raghavendra G (rgowdapp) 
------
commit 91c9f4a19fde4894576b398252c77f730832a26a
Author: N Balachandran <nbalacha>
Date:   Mon Jul 31 12:32:59 2017 +0530

    cluster/dht: Check for open fd only on EBADF
    
    DHT fd based fops will now check if the fd is
    open on the cached subvol only if the call fails
    with EBADF.
    
    This will improve performance for scenarios where
    a rebalance is not running which would be most of
    the time.
    
    Change-Id: Idfaeb8927af769c6110d07a165a0fe2307369239
    BUG: 1476665
    Signed-off-by: N Balachandran <nbalacha>
    Reviewed-on: https://review.gluster.org/17922
    Smoke: Gluster Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>
    Reviewed-by: Raghavendra G <rgowdapp>

--- Additional comment from Worker Ant on 2017-08-03 23:26:26 EDT ---

REVIEW: https://review.gluster.org/17970 (Revert "cluster/dht: Check for open fd only on EBADF") posted (#1) for review on master by N Balachandran (nbalacha)

--- Additional comment from Worker Ant on 2017-08-04 04:56:25 EDT ---

COMMIT: https://review.gluster.org/17970 committed in master by N Balachandran (nbalacha) 
------
commit 6d1068ddb35be19df36210c9fcaa7ce97e2a376a
Author: N Balachandran <nbalacha>
Date:   Fri Aug 4 08:52:31 2017 +0530

    Revert "cluster/dht: Check for open fd only on EBADF"
    
    This reverts commit 91c9f4a19fde4894576b398252c77f730832a26a.
    This patch needs to be reworked.
    
    Change-Id: I4c24f647c2b1abc68fc4e9fe6eb810418e2033aa
    BUG: 1476665
    Signed-off-by: N Balachandran <nbalacha>
    Reviewed-on: https://review.gluster.org/17970
    Smoke: Gluster Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>

--- Additional comment from Worker Ant on 2017-08-04 05:21:40 EDT ---

REVIEW: https://review.gluster.org/17976 (cluster/dht: Check for open fd only on EBADF) posted (#1) for review on master by N Balachandran (nbalacha)

--- Additional comment from Worker Ant on 2017-08-04 05:35:31 EDT ---

REVIEW: https://review.gluster.org/17976 (cluster/dht: Check for open fd only on EBADF) posted (#2) for review on master by N Balachandran (nbalacha)

--- Additional comment from Worker Ant on 2017-08-08 06:21:28 EDT ---

COMMIT: https://review.gluster.org/17976 committed in master by Raghavendra G (rgowdapp) 
------
commit cdca1cb26a0aba390c6d8485c0d6d95e22ffc8bd
Author: N Balachandran <nbalacha>
Date:   Fri Aug 4 14:46:38 2017 +0530

    cluster/dht: Check for open fd only on EBADF
    
    DHT fd based fops used to check if the fd was open
    on the cached subvol before winding the call. However,
    this introduced a performance regression of about
    30% for reads.
    
    This check was introduced to handle cases where files
    were migrated while IOs were happening. As this is not
    the common case, dht will now check if the fd is
    open on the cached subvol only if the call fails
    with EBADF.
    
    This will prevent a performance hit where a rebalance
    is not running.
    
    Change-Id: I2035a858d63c3fcd22bb634055bbb0ad01686808
    BUG: 1476665
    Signed-off-by: N Balachandran <nbalacha>
    Reviewed-on: https://review.gluster.org/17976
    Smoke: Gluster Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.org>
    Reviewed-by: Amar Tumballi <amarts>
    Reviewed-by: Susant Palai <spalai>
    Reviewed-by: Raghavendra G <rgowdapp>