Bug 1202212
Summary: | Performance enhancement for RDMA | ||
---|---|---|---|
Product: | [Community] GlusterFS | Reporter: | Mohammed Rafi KC <rkavunga> |
Component: | rdma | Assignee: | bugs <bugs> |
Status: | CLOSED CURRENTRELEASE | QA Contact: | |
Severity: | medium | Docs Contact: | |
Priority: | high | ||
Version: | 3.6.0 | CC: | bugs, chrisw, hchiramm, nlevinki, rabhat, rwheeler, storage-qa-internal, vagarwal |
Target Milestone: | --- | Keywords: | Triaged |
Target Release: | --- | ||
Hardware: | x86_64 | ||
OS: | Linux | ||
Whiteboard: | |||
Fixed In Version: | glusterfs-v3.6.3 | Doc Type: | Bug Fix |
Doc Text: | Story Points: | --- | |
Clone Of: | 1187456 | Environment: | |
Last Closed: | 2016-02-04 15:21:33 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | 1186127, 1187456 | ||
Bug Blocks: | 1184460 |
Description
Mohammed Rafi KC
2015-03-16 06:31:58 UTC
REVIEW: http://review.gluster.org/9886 (rdma : agregate a vectored read as one) posted (#1) for review on release-3.6 by mohammed rafi kc (rkavunga) REVIEW: http://review.gluster.org/9887 (rdma: post multiple work request in a single call.) posted (#1) for review on release-3.6 by mohammed rafi kc (rkavunga) REVIEW: http://review.gluster.org/9888 (rdma:read multiple wr from cq and ack them in one call) posted (#1) for review on release-3.6 by mohammed rafi kc (rkavunga) REVIEW: http://review.gluster.org/9889 (rdma: pre-register iobuf_pool with rdma devices.) posted (#1) for review on release-3.6 by mohammed rafi kc (rkavunga) REVIEW: http://review.gluster.org/9890 (rdma:setting wrong remote memory.) posted (#1) for review on release-3.6 by mohammed rafi kc (rkavunga) REVIEW: http://review.gluster.org/9891 (rdma:changing list iteration to safe mode) posted (#1) for review on release-3.6 by mohammed rafi kc (rkavunga) COMMIT: http://review.gluster.org/9886 committed in release-3.6 by Raghavendra Bhat (raghavendra) ------ commit 692f1c6c92152fb592b0c35c0faa1610dbecaad5 Author: Mohammed Rafi KC <rkavunga> Date: Mon Dec 22 11:04:50 2014 +0530 rdma : agregate a vectored read as one Back port of : http://review.gluster.org/9321 For a vectored read with payload count>1 will make two read requests and to hold that a single contiguous memory is allocated. So after completing the read request, instead of sending as vector we will aggregate all the reads one. Change-Id: I15e7d7bddc1a62d5097a39392575f47cfff3d3a8 BUG: 1202212 Signed-off-by: Mohammed Rafi KC <rkavunga> Reviewed-on: http://review.gluster.org/9321 Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Raghavendra G <rgowdapp> Tested-by: Raghavendra G <rgowdapp> Reviewed-on: http://review.gluster.org/9886 Reviewed-by: Kaleb KEITHLEY <kkeithle> Reviewed-by: Raghavendra Bhat <raghavendra> COMMIT: http://review.gluster.org/9887 committed in release-3.6 by Raghavendra Bhat (raghavendra) ------ commit 8f12ed326db9e56e993947174b5048fca79dfc42 Author: Mohammed Rafi KC <rkavunga> Date: Mon Dec 22 15:16:43 2014 +0530 rdma: post multiple work request in a single call. Back port of : http://review.gluster.org/9327 ibv_post-send will allow to send multiple work request in a single call posting as linked list. So if the payload count > 1, we can perform the data operation in a single call to ibv_post_send. Change-Id: Ib2e485cbbe6887919109e73e17d4fab595d5e65e BUG: 1202212 Signed-off-by: Mohammed Rafi KC <rkavunga> Reviewed-on: http://review.gluster.org/9327 Reviewed-by: Raghavendra G <rgowdapp> Tested-by: Raghavendra G <rgowdapp> Reviewed-on: http://review.gluster.org/9887 Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Kaleb KEITHLEY <kkeithle> Reviewed-by: Raghavendra Bhat <raghavendra> COMMIT: http://review.gluster.org/9888 committed in release-3.6 by Raghavendra Bhat (raghavendra) ------ commit d21990e093d99d8adbacae1ba2c56ff7606e2c37 Author: Mohammed Rafi KC <rkavunga> Date: Tue Dec 23 11:31:37 2014 +0530 rdma:read multiple wr from cq and ack them in one call Back port of : http://review.gluster.org/9329 we are reading one work completion request at a time from cq, though we can read multiple work completion requests from cq. Also we can acknowledge them in one call itself. Both will give a better performance because of less mutual exclusion locks are being performed. Change-Id: Ib5664cab25c87db7f575d482eee4dcd2b5005c04 BUG: 1202212 Signed-off-by: Mohammed Rafi KC <rkavunga> Reviewed-on: http://review.gluster.org/9329 Reviewed-by: Raghavendra G <rgowdapp> Tested-by: Raghavendra G <rgowdapp> Reviewed-on: http://review.gluster.org/9888 Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Kaleb KEITHLEY <kkeithle> Reviewed-by: Raghavendra Bhat <raghavendra> COMMIT: http://review.gluster.org/9889 committed in release-3.6 by Raghavendra Bhat (raghavendra) ------ commit 7febb66a26f01c94f8e76bb90cf4edd7c6cc1421 Author: Mohammed Rafi KC <rkavunga> Date: Tue Feb 17 20:17:58 2015 +0530 rdma: pre-register iobuf_pool with rdma devices. Back port pf : http://review.gluster.org/9506 registring buffers with rdma device is a time consuming operation. So performing registration in code path will decrease the performance. Using a pre registered memory will give a bettor performance, ie, register iobuf_pool during rdma initialization. For dynamically created arena, we can register with all the device. Change-Id: Ic79183e2efd014c43faf5911fdb6d5cfbcee64ca BUG: 1202212 Signed-off-by: Mohammed Rafi KC <rkavunga> Reviewed-on: http://review.gluster.org/9506 Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Raghavendra G <rgowdapp> Tested-by: Raghavendra G <rgowdapp> Reviewed-on: http://review.gluster.org/9889 Reviewed-by: Raghavendra Bhat <raghavendra> COMMIT: http://review.gluster.org/9890 committed in release-3.6 by Raghavendra Bhat (raghavendra) ------ commit 7095d05ae3e06c2ea0f01b18cb117a817316fb9e Author: Mohammed Rafi KC <rkavunga> Date: Wed Mar 4 14:37:05 2015 +0530 rdma:setting wrong remote memory. Back port of : http://review.gluster.org/9794 when we send more than one work request in a single call, the remote addr is always setting as the first address of the vector. Change-Id: I55aea7bd6542abe22916719a139f7c8f73334d26 BUG: 1202212 Signed-off-by: Mohammed Rafi KC <rkavunga> Reviewed-on: http://review.gluster.org/9794 Reviewed-by: Raghavendra G <rgowdapp> Tested-by: Raghavendra G <rgowdapp> Reviewed-on: http://review.gluster.org/9890 Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Kaleb KEITHLEY <kkeithle> Reviewed-by: Raghavendra Bhat <raghavendra> COMMIT: http://review.gluster.org/9891 committed in release-3.6 by Raghavendra Bhat (raghavendra) ------ commit d5b6c5265524efe8f927066f342b89f12a531079 Author: Mohammed Rafi KC <rkavunga> Date: Fri Mar 13 11:37:14 2015 +0530 rdma:changing list iteration to safe mode Bck port of : http://review.gluster.org/9872 Change-Id: I2299378f02a5577a8bf2874664ba79e92c3811b5 BUG: 1202212 Signed-off-by: Mohammed Rafi KC <rkavunga> Reviewed-on: http://review.gluster.org/9872 Reviewed-by: Krishnan Parthasarathi <kparthas> Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Raghavendra Talur <rtalur> Reviewed-by: Vijay Bellur <vbellur> Reviewed-on: http://review.gluster.org/9891 Reviewed-by: Kaleb KEITHLEY <kkeithle> Reviewed-by: Raghavendra Bhat <raghavendra> This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-v3.6.3, please open a new bug report. glusterfs-v3.6.3 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] https://www.gluster.org/pipermail/gluster-users/2015-April/021669.html [2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user |