Description of problem: run fio to perform test in thread mode will cause gfapi to crash. Digging into this issue, and found that glfs_set_logging changes THIS->ctx without lock, during initialization "THIS" is actually global_xlator. Concurrently changing it could result in global_xlator.ctx be changed to some fs->ctx. Consider the following sequence. thread1 thread2 old_ctx = THIS->ctx // current old_ctx is global_xlator.ctx THIS->ctx = fs1->ctx old_ctx = THIS->ctx // it is fs1->ctx actually THIS->ctx = fs2->ctx THIS->ctx = old_ctx // revert back to global_xlator.ctx THIS->ctx = old_ctx // THIS->ctx becomes fs1->ctx global_xlator.ctx might refers to an invalid pointer and that causes subsequent new fs init crash as they might use global_xlator. Version-Release number of selected component (if applicable): How reproducible: run fio Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
REVIEW: https://review.gluster.org/18940 (libglusterfs: specify ctx in gf_log_set_loglevel) posted (#1) for review on master by Zhang Huan
REVIEW: https://review.gluster.org/18941 (gfapi: fix issue when glfs_set_logging is called concurrently) posted (#1) for review on master by Zhang Huan
COMMIT: https://review.gluster.org/18940 committed in master by \"Zhang Huan\" <zhanghuan> with a commit message- libglusterfs: specify ctx in gf_log_set_loglevel specify ctx in gf_log_set_loglevel, instead of getting it from a thread specific variable. Change-Id: I498f826e8e32231235a6b0005026a27c327727fd BUG: 1521213 Signed-off-by: Zhang Huan <zhanghuan>
COMMIT: https://review.gluster.org/18941 committed in master by \"Zhang Huan\" <zhanghuan> with a commit message- gfapi: fix issue when glfs_set_logging is called concurrently glfs_set_logging changes THIS->ctx without lock, during initialization THIS refers to global_xlator. Concurrently changing it could result in global_xlator.ctx being some fs->ctx. After that fs->ctx is destroyed, global_xlator.ctx refers to an invalid pointer and that causes crash in sbusequent access. Change-Id: I6682173811799bafd525d9ab371874f3803baa0e BUG: 1521213 Signed-off-by: Zhang Huan <zhanghuan>
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-4.0.0, please open a new bug report. glusterfs-4.0.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://lists.gluster.org/pipermail/announce/2018-March/000092.html [2] https://www.gluster.org/pipermail/gluster-users/