Back to bug 2178838

Who When What Removed Added
Red Hat One Jira (issues.redhat.com) 2023-03-15 20:56:51 UTC Link ID Red Hat Issue Tracker RHCEPH-6263
Brett Hull 2023-03-15 21:12:19 UTC CC bhull
Raimund Sacherer 2023-03-16 14:49:14 UTC CC rsachere
Ilya Dryomov 2023-03-16 18:27:03 UTC Flags needinfo?(bhubbard)
CC bhubbard
Brad Hubbard 2023-03-17 03:00:04 UTC Flags needinfo?(bhubbard)
Ilya Dryomov 2023-03-17 09:32:08 UTC Assignee idryomov xiubli
Target Release 6.1z1 5.3z2
Status NEW ASSIGNED
CC idryomov
loberman 2023-03-17 12:21:00 UTC CC xiubli
Flags needinfo?(xiubli)
Xiubo Li 2023-03-17 14:00:51 UTC Flags needinfo?(xiubli)
Xiubo Li 2023-03-18 04:39:55 UTC Flags needinfo?(rsachere)
Xiubo Li 2023-03-20 03:00:45 UTC Link ID Github open-iscsi/tcmu-runner/pull/695
Ilya Dryomov 2023-03-21 13:10:39 UTC Flags needinfo?(pnataraj)
CC pnataraj
Veera Raghava Reddy 2023-03-21 14:43:30 UTC CC vereddy
Flags needinfo?(pnataraj)
Raimund Sacherer 2023-03-21 15:12:57 UTC Flags needinfo?(rsachere)
Xiubo Li 2023-03-22 03:01:08 UTC Flags needinfo?(idryomov)
Brad Hubbard 2023-03-22 03:13:12 UTC Flags needinfo?(idryomov)
Raimund Sacherer 2023-03-23 09:50:53 UTC Flags needinfo?(xiubli)
Ilya Dryomov 2023-03-23 10:44:13 UTC Flags needinfo?(xiubli)
Ilya Dryomov 2023-03-23 14:29:23 UTC Flags needinfo?(tserlin)
Status ASSIGNED POST
CC tserlin
Brad Hubbard 2023-03-24 04:02:05 UTC Flags needinfo?(tserlin)
Flags needinfo?(tserlin)
Raimund Sacherer 2023-03-24 07:32:00 UTC Flags needinfo? needinfo?(idryomov)
Ilya Dryomov 2023-03-24 09:44:54 UTC Flags needinfo?(idryomov)
Raimund Sacherer 2023-03-24 16:48:37 UTC Summary CEPH 5.2 iscsi gateway tcmu-runner tasks are segfaulting and client connections are lost (ceph-base-16.2.7-112.el8cp.x86_64) CEPH 5.1 iscsi gateway tcmu-runner tasks are segfaulting and client connections are lost (ceph-base-16.2.7-112.el8cp.x86_64)
Ilya Dryomov 2023-03-27 10:32:30 UTC Flags needinfo?(tserlin)
Status POST MODIFIED
CC rcyriac
Flags needinfo?(rcyriac)
Flags needinfo?
loberman 2023-03-28 12:46:23 UTC Flags needinfo?(xiubli)
Michael J. Kidd 2023-03-28 14:44:33 UTC CC linuxkidd
Michael J. Kidd 2023-03-28 16:05:25 UTC Flags needinfo?(tserlin)
Xiubo Li 2023-03-29 00:59:54 UTC Flags needinfo?(tserlin)
Flags needinfo?(linuxkidd)
Flags needinfo?(xiubli)
Bipin Kunal 2023-03-29 06:26:43 UTC Flags needinfo?(loberman)
CC bkunal
loberman 2023-03-29 12:17:48 UTC Flags needinfo?(loberman) needinfo?(bkunal)
Bipin Kunal 2023-03-29 14:10:53 UTC Flags needinfo?(bkunal)
errata-xmlrpc 2023-03-29 17:00:56 UTC Status MODIFIED ON_QA
Rejy M Cyriac 2023-03-29 17:53:08 UTC Flags needinfo?(rcyriac)
loberman 2023-03-29 20:35:39 UTC Flags needinfo?(bkunal)
Bipin Kunal 2023-03-30 09:27:08 UTC Flags needinfo?(bkunal) needinfo?(loberman)
loberman 2023-03-30 11:15:15 UTC Flags needinfo?(loberman)
Manny 2023-04-02 15:00:30 UTC Flags needinfo?(linuxkidd)
CC mcaldeir
Brad Hubbard 2023-04-02 21:50:00 UTC Flags needinfo?(ktdreyer)
CC ktdreyer
Preethi 2023-04-03 06:25:38 UTC Flags needinfo?(xiubli)
Preethi 2023-04-03 06:26:16 UTC Flags needinfo?(tserlin)
Xiubo Li 2023-04-03 11:59:01 UTC Flags needinfo?(xiubli) needinfo?(tserlin)
Preethi 2023-04-04 17:04:54 UTC Flags needinfo?(tserlin)
Flags needinfo?(tserlin)
Status ON_QA VERIFIED
Geo Jose 2023-04-06 08:47:36 UTC CC gjose
Akash Raj 2023-04-10 14:31:39 UTC CC akraj
Flags needinfo?(xiubli)
Docs Contact akraj
Xiubo Li 2023-04-11 00:47:23 UTC Doc Text Cause:

There is a racy between reopening devices and reporting events to ceph server.

Consequence:

It will cause use-after-free crash bug.

Fix:

To make sure all the in-flight IOs have been finished before flushing the event_work. Or just after we flush the event_work a new timedout IO callback could come and then it will fire a new event work, which may access the device while we are closing the device later.

Result:

No crash happen any more.
Flags needinfo?(xiubli)
Doc Type If docs needed, set a value Bug Fix
Akash Raj 2023-04-11 05:37:46 UTC Blocks 2185621
Akash Raj 2023-04-11 07:24:11 UTC Doc Text Cause:

There is a racy between reopening devices and reporting events to ceph server.

Consequence:

It will cause use-after-free crash bug.

Fix:

To make sure all the in-flight IOs have been finished before flushing the event_work. Or just after we flush the event_work a new timedout IO callback could come and then it will fire a new event work, which may access the device while we are closing the device later.

Result:

No crash happen any more.
.`use-after-free` bug crash no longer occurs

Previously, a racy condition between reopening devices and reporting events to the Ceph server caused _use-after-free_ bug crash.

With this fix, to ensure that all the in-flight IOs are finished before or after flushing the `event_work`, a new timedout IO callback is implemented. This fires a new `event_work` which might access the device while closing the device and there are no more crashes.
errata-xmlrpc 2023-04-11 16:06:44 UTC Status VERIFIED RELEASE_PENDING
errata-xmlrpc 2023-04-11 20:07:59 UTC Status RELEASE_PENDING CLOSED
Resolution --- ERRATA
Last Closed 2023-04-11 20:07:59 UTC
errata-xmlrpc 2023-04-11 20:08:18 UTC Link ID Red Hat Product Errata RHBA-2023:1732
Manny 2023-04-17 16:24:44 UTC Link ID Red Hat Knowledge Base (Solution) 7008129
Ranjini M N 2023-05-19 07:10:58 UTC Blocks 2203283
Blocks 2203283

Back to bug 2178838