Bug 1202212 - Performance enhancement for RDMA
Summary: Performance enhancement for RDMA
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: rdma
Version: 3.6.0
Hardware: x86_64
OS: Linux
high
medium
Target Milestone: ---
Assignee: bugs@gluster.org
QA Contact:
URL:
Whiteboard:
Depends On: 1186127 1187456
Blocks: glusterfs-3.6.3
TreeView+ depends on / blocked
 
Reported: 2015-03-16 06:31 UTC by Mohammed Rafi KC
Modified: 2016-02-04 15:21 UTC (History)
8 users (show)

Fixed In Version: glusterfs-v3.6.3
Clone Of: 1187456
Environment:
Last Closed: 2016-02-04 15:21:33 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)

Description Mohammed Rafi KC 2015-03-16 06:31:58 UTC
+++ This bug was initially created as a clone of Bug #1187456 +++

+++ This bug was initially created as a clone of Bug #1186127 +++

Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

--- Additional comment from Anand Avati on 2015-01-30 00:12:39 EST ---

REVIEW: http://review.gluster.org/9506 (rdma: pre-register iobuf_pool with rdma devices.) posted (#1) for review on master by mohammed rafi  kc (rkavunga)

--- Additional comment from Anand Avati on 2015-02-17 09:50:59 EST ---

REVIEW: http://review.gluster.org/9506 (rdma: pre-register iobuf_pool with rdma devices.) posted (#2) for review on master by Raghavendra G (rgowdapp)

--- Additional comment from Anand Avati on 2015-02-17 10:57:52 EST ---

REVIEW: http://review.gluster.org/9506 (rdma: pre-register iobuf_pool with rdma devices.) posted (#3) for review on master by Raghavendra G (rgowdapp)

--- Additional comment from Anand Avati on 2015-02-17 23:10:03 EST ---

COMMIT: http://review.gluster.org/9506 committed in master by Raghavendra G (rgowdapp) 
------
commit abcb2017b07b0706aef0d9eeb729115ca605ad6d
Author: Mohammed Rafi KC <rkavunga>
Date:   Tue Feb 17 20:17:58 2015 +0530

    rdma: pre-register iobuf_pool with rdma devices.
    
    registring buffers with rdma device is a time consuming
    operation. So performing registration in code path will
    decrease the performance.
    Using a pre registered memory will give a bettor performance,
    ie, register iobuf_pool during rdma initialization. For
    dynamically created arena, we can register with all the
    device.
    
    Change-Id: Ic79183e2efd014c43faf5911fdb6d5cfbcee64ca
    BUG: 1187456
    Signed-off-by: Mohammed Rafi KC <rkavunga>
    Reviewed-on: http://review.gluster.org/9506
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Raghavendra G <rgowdapp>
    Tested-by: Raghavendra G <rgowdapp>

--- Additional comment from Anand Avati on 2015-02-19 06:47:49 EST ---

REVIEW: http://review.gluster.org/9697 (rdma: Free resources related to iobuf in fini) posted (#3) for review on master by mohammed rafi  kc (rkavunga)

--- Additional comment from Anand Avati on 2015-02-19 08:55:33 EST ---

REVIEW: http://review.gluster.org/9697 (rdma: Free resources related to iobuf in fini) posted (#4) for review on master by mohammed rafi  kc (rkavunga)

--- Additional comment from Anand Avati on 2015-03-10 22:28:40 EDT ---

COMMIT: http://review.gluster.org/9697 committed in master by Vijay Bellur (vbellur) 
------
commit 219512c5014e9a13081d6a9981ae02b54586d801
Author: Mohammed Rafi KC <rkavunga>
Date:   Thu Feb 19 15:52:28 2015 +0530

    rdma: Free resources related to iobuf in fini
    
    If rdma transport is destroyed because of any reason,
    then rdma.so will be unloaded. But we are not setting
    iobuf registeration function to null. After this, if
    an iobuf request is came, then we will try to call a
    function which is not loaded.
    
    Change-Id: I3293f9974e16d8e865131785ee697ea02be8cdfc
    BUG: 1187456
    Signed-off-by: Mohammed Rafi KC <rkavunga>
    Reviewed-on: http://review.gluster.org/9697
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Raghavendra Talur <rtalur>
    Reviewed-by: Vijay Bellur <vbellur>

Comment 1 Anand Avati 2015-03-16 06:33:32 UTC
REVIEW: http://review.gluster.org/9886 (rdma : agregate a vectored read as one) posted (#1) for review on release-3.6 by mohammed rafi  kc (rkavunga)

Comment 2 Anand Avati 2015-03-16 06:33:35 UTC
REVIEW: http://review.gluster.org/9887 (rdma: post multiple work request in a single call.) posted (#1) for review on release-3.6 by mohammed rafi  kc (rkavunga)

Comment 3 Anand Avati 2015-03-16 06:33:38 UTC
REVIEW: http://review.gluster.org/9888 (rdma:read multiple wr from cq and ack them in one call) posted (#1) for review on release-3.6 by mohammed rafi  kc (rkavunga)

Comment 4 Anand Avati 2015-03-16 06:33:42 UTC
REVIEW: http://review.gluster.org/9889 (rdma: pre-register iobuf_pool with rdma devices.) posted (#1) for review on release-3.6 by mohammed rafi  kc (rkavunga)

Comment 5 Anand Avati 2015-03-16 06:33:45 UTC
REVIEW: http://review.gluster.org/9890 (rdma:setting wrong remote memory.) posted (#1) for review on release-3.6 by mohammed rafi  kc (rkavunga)

Comment 6 Anand Avati 2015-03-16 06:33:48 UTC
REVIEW: http://review.gluster.org/9891 (rdma:changing list iteration to safe mode) posted (#1) for review on release-3.6 by mohammed rafi  kc (rkavunga)

Comment 7 Anand Avati 2015-03-27 11:33:15 UTC
COMMIT: http://review.gluster.org/9886 committed in release-3.6 by Raghavendra Bhat (raghavendra) 
------
commit 692f1c6c92152fb592b0c35c0faa1610dbecaad5
Author: Mohammed Rafi KC <rkavunga>
Date:   Mon Dec 22 11:04:50 2014 +0530

    rdma : agregate a vectored read as one
    
            Back port of : http://review.gluster.org/9321
    
    For a vectored read with payload count>1 will make two read
    requests and to hold that a single contiguous memory is allocated.
    So after completing the read request, instead of sending as vector
    we will aggregate all the reads one.
    
    Change-Id: I15e7d7bddc1a62d5097a39392575f47cfff3d3a8
    BUG: 1202212
    Signed-off-by: Mohammed Rafi KC <rkavunga>
    Reviewed-on: http://review.gluster.org/9321
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Raghavendra G <rgowdapp>
    Tested-by: Raghavendra G <rgowdapp>
    Reviewed-on: http://review.gluster.org/9886
    Reviewed-by: Kaleb KEITHLEY <kkeithle>
    Reviewed-by: Raghavendra Bhat <raghavendra>

Comment 8 Anand Avati 2015-03-27 11:33:38 UTC
COMMIT: http://review.gluster.org/9887 committed in release-3.6 by Raghavendra Bhat (raghavendra) 
------
commit 8f12ed326db9e56e993947174b5048fca79dfc42
Author: Mohammed Rafi KC <rkavunga>
Date:   Mon Dec 22 15:16:43 2014 +0530

    rdma: post multiple work request in a single call.
    
            Back port of : http://review.gluster.org/9327
    
    ibv_post-send will allow to send multiple work request
    in a single call posting as linked list.
    
    So if the payload count > 1, we can perform the data
    operation in a single call to ibv_post_send.
    
    Change-Id: Ib2e485cbbe6887919109e73e17d4fab595d5e65e
    BUG: 1202212
    Signed-off-by: Mohammed Rafi KC <rkavunga>
    Reviewed-on: http://review.gluster.org/9327
    Reviewed-by: Raghavendra G <rgowdapp>
    Tested-by: Raghavendra G <rgowdapp>
    Reviewed-on: http://review.gluster.org/9887
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Kaleb KEITHLEY <kkeithle>
    Reviewed-by: Raghavendra Bhat <raghavendra>

Comment 9 Anand Avati 2015-03-27 11:33:51 UTC
COMMIT: http://review.gluster.org/9888 committed in release-3.6 by Raghavendra Bhat (raghavendra) 
------
commit d21990e093d99d8adbacae1ba2c56ff7606e2c37
Author: Mohammed Rafi KC <rkavunga>
Date:   Tue Dec 23 11:31:37 2014 +0530

    rdma:read multiple wr from cq and ack them in one call
    
            Back port of : http://review.gluster.org/9329
    
    we are reading one work completion request at a time
    from cq, though we can read multiple work completion
    requests from cq. Also we can acknowledge them in
    one call itself. Both will give a better performance
    because of less mutual exclusion locks are being performed.
    
    Change-Id: Ib5664cab25c87db7f575d482eee4dcd2b5005c04
    BUG: 1202212
    Signed-off-by: Mohammed Rafi KC <rkavunga>
    Reviewed-on: http://review.gluster.org/9329
    Reviewed-by: Raghavendra G <rgowdapp>
    Tested-by: Raghavendra G <rgowdapp>
    Reviewed-on: http://review.gluster.org/9888
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Kaleb KEITHLEY <kkeithle>
    Reviewed-by: Raghavendra Bhat <raghavendra>

Comment 10 Anand Avati 2015-03-27 11:34:12 UTC
COMMIT: http://review.gluster.org/9889 committed in release-3.6 by Raghavendra Bhat (raghavendra) 
------
commit 7febb66a26f01c94f8e76bb90cf4edd7c6cc1421
Author: Mohammed Rafi KC <rkavunga>
Date:   Tue Feb 17 20:17:58 2015 +0530

    rdma: pre-register iobuf_pool with rdma devices.
    
            Back port pf : http://review.gluster.org/9506
    
    registring buffers with rdma device is a time consuming
    operation. So performing registration in code path will
    decrease the performance.
    Using a pre registered memory will give a bettor performance,
    ie, register iobuf_pool during rdma initialization. For
    dynamically created arena, we can register with all the
    device.
    
    Change-Id: Ic79183e2efd014c43faf5911fdb6d5cfbcee64ca
    BUG: 1202212
    Signed-off-by: Mohammed Rafi KC <rkavunga>
    Reviewed-on: http://review.gluster.org/9506
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Raghavendra G <rgowdapp>
    Tested-by: Raghavendra G <rgowdapp>
    Reviewed-on: http://review.gluster.org/9889
    Reviewed-by: Raghavendra Bhat <raghavendra>

Comment 11 Anand Avati 2015-03-27 11:34:43 UTC
COMMIT: http://review.gluster.org/9890 committed in release-3.6 by Raghavendra Bhat (raghavendra) 
------
commit 7095d05ae3e06c2ea0f01b18cb117a817316fb9e
Author: Mohammed Rafi KC <rkavunga>
Date:   Wed Mar 4 14:37:05 2015 +0530

    rdma:setting wrong remote memory.
    
            Back port of : http://review.gluster.org/9794
    
    when we send more than one work request in a single call,
    the remote addr is always setting as the first address of
    the vector.
    
    Change-Id: I55aea7bd6542abe22916719a139f7c8f73334d26
    BUG: 1202212
    Signed-off-by: Mohammed Rafi KC <rkavunga>
    Reviewed-on: http://review.gluster.org/9794
    Reviewed-by: Raghavendra G <rgowdapp>
    Tested-by: Raghavendra G <rgowdapp>
    Reviewed-on: http://review.gluster.org/9890
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Kaleb KEITHLEY <kkeithle>
    Reviewed-by: Raghavendra Bhat <raghavendra>

Comment 12 Anand Avati 2015-03-27 11:35:35 UTC
COMMIT: http://review.gluster.org/9891 committed in release-3.6 by Raghavendra Bhat (raghavendra) 
------
commit d5b6c5265524efe8f927066f342b89f12a531079
Author: Mohammed Rafi KC <rkavunga>
Date:   Fri Mar 13 11:37:14 2015 +0530

    rdma:changing list iteration to safe mode
    
            Bck port of : http://review.gluster.org/9872
    
    Change-Id: I2299378f02a5577a8bf2874664ba79e92c3811b5
    BUG: 1202212
    Signed-off-by: Mohammed Rafi KC <rkavunga>
    Reviewed-on: http://review.gluster.org/9872
    Reviewed-by: Krishnan Parthasarathi <kparthas>
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Raghavendra Talur <rtalur>
    Reviewed-by: Vijay Bellur <vbellur>
    Reviewed-on: http://review.gluster.org/9891
    Reviewed-by: Kaleb KEITHLEY <kkeithle>
    Reviewed-by: Raghavendra Bhat <raghavendra>

Comment 13 Kaushal 2016-02-04 15:21:33 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-v3.6.3, please open a new bug report.

glusterfs-v3.6.3 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] https://www.gluster.org/pipermail/gluster-users/2015-April/021669.html
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user


Note You need to log in before you can comment on or make changes to this bug.