Bug 1522651
| Summary: | rdma transport may access an obsolete item in gf_rdma_device_t->all_mr, and causes glusterfsd/glusterfs process crash. | |||
|---|---|---|---|---|
| Product: | [Community] GlusterFS | Reporter: | Yi Wang <wangyi> | |
| Component: | rdma | Assignee: | Mohammed Rafi KC <rkavunga> | |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Rahul Hinduja <rhinduja> | |
| Severity: | urgent | Docs Contact: | ||
| Priority: | unspecified | |||
| Version: | mainline | CC: | bugs, rhs-bugs, rwheeler | |
| Target Milestone: | --- | Keywords: | Bugfix | |
| Target Release: | --- | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | glusterfs-4.0.0 | Doc Type: | No Doc Update | |
| Doc Text: |
undefined
|
Story Points: | --- | |
| Clone Of: | ||||
| : | 1525850 1527699 (view as bug list) | Environment: | ||
| Last Closed: | 2018-03-15 11:22:36 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1525850, 1527699 | |||
correct product name to GlusterFS. correct the hardware selection. COMMIT: https://review.gluster.org/18943 committed in master by \"Yi Wang\" <wangyi> with a commit message- rpc-transport/rdma: Add a mutex for the list of RDMA Memory Region(MR) access Problem: gf_rdma_device_t->all_mr is a __gf_rdma_arena_mr(includes MR content) kind of list in the rdma rpc-transport. The rdma rpc-transport will add/delete items to the list when MRs register, deregister, and free. Because gf_rdma_device_t->all_mr is used by different threads and it is not mutex protected, rdma transport maybe access obsolete items in it. Solution: Add a mutex protection for the gf_rdma_device_t->all_mr. Change-Id: I2b7de0f7aa516b90bb6f3c6aae3aadd23b243900 BUG: 1522651 Signed-off-by: Yi Wang <wangyi> REVIEW: https://review.gluster.org/19032 (rpc-transport/rdma: Add a mutex for the list of RDMA Memory Region(MR) access) posted (#1) for review on release-3.12 by Yi Wang REVIEW: https://review.gluster.org/19033 (rpc-transport/rdma: Add a mutex for the list of RDMA Memory Region(MR) access) posted (#1) for review on release-3.13 by Yi Wang The bugfix of this bug needs to backport to release-3.11~3.13. And the commits depend on the bug status. Please help me change the bug status.
>> BUG id 1522651 has an invalid status as MODIFIED. Acceptable status values are NEW, ASSIGNED or POST.
REVISION POSTED: https://review.gluster.org/19032 (rpc-transport/rdma: Add a mutex for the list of RDMA Memory Region(MR) access) posted (#2) for review on release-3.12 by mohammed rafi kc REVIEW: https://review.gluster.org/19035 (rpc-transport/rdma: Add a mutex for the list of RDMA Memory Region(MR) access) posted (#1) for review on release-3.11 by Yi Wang REVISION POSTED: https://review.gluster.org/19033 (rpc-transport/rdma: Add a mutex for the list of RDMA Memory Region(MR) access) posted (#2) for review on release-3.13 by Shyamsundar Ranganathan This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-4.0.0, please open a new bug report. glusterfs-4.0.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://lists.gluster.org/pipermail/announce/2018-March/000092.html [2] https://www.gluster.org/pipermail/gluster-users/ |
Description of problem: In the rdma.c file, gf_rdma_device_t->all_mr is a __gf_rdma_arena_mr(include RDMA Memory Region(MR) content) kind of list in the rdma rpc-transport. The rdma rpc-transport will add/delete items to the gf_rdma_device_t->all_mr when MRs register, deregister, and free. Because gf_rdma_device_t->all_mr is used by different threads and it is not mutex protected, rdma transport maybe access obsolete items in it. Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: At a heavy condition, the items in the gf_rdma_device_t->all_mr should be released by threads. As a result, glusterfsd/glusterfs process will crash. Expected results: gf_rdma_device_t->all_mr must be mutex protected. Additional info: None