Back to bug 2158304

Who When What Removed Added
Red Hat One Jira (issues.redhat.com) 2023-01-05 03:31:23 UTC Link ID Red Hat Issue Tracker RHCEPH-5885
Xiubo Li 2023-01-05 03:48:17 UTC Link ID Ceph Project Bug Tracker 58000
Xiubo Li 2023-01-05 03:49:29 UTC Severity unspecified low
Xiubo Li 2023-01-05 03:51:29 UTC Status NEW ASSIGNED
Greg Farnum 2023-01-05 14:57:12 UTC Assignee nobody xiubli
CC gfarnum
Red Hat Bugzilla 2023-01-09 08:29:49 UTC CC ceph-eng-bugs
Alasdair Kergon 2023-01-09 19:43:36 UTC CC ceph-eng-bugs
Xiubo Li 2023-04-10 13:40:29 UTC Status ASSIGNED MODIFIED
Keywords Rebase
Hemanth Kumar 2023-04-12 07:10:33 UTC CC hyelloji
Hemanth Kumar 2023-04-12 07:10:51 UTC QA Contact hyelloji
errata-xmlrpc 2023-07-11 20:46:38 UTC Fixed In Version ceph-17.2.6-84.el9cp
CC tserlin
Status MODIFIED ON_QA
Akash Raj 2023-07-13 17:36:52 UTC CC akraj
Flags needinfo?(xiubli)
Blocks 2221020
Docs Contact akraj
Hemanth Kumar 2023-07-13 18:09:56 UTC Status ON_QA VERIFIED
Xiubo Li 2023-07-14 00:51:54 UTC Flags needinfo?(xiubli)
Doc Text Feature:

Switch the unfair Mutex lock to fair mutex.


Reason:

The implementations of the Mutex (e.g. std::mutex in C++) do not
guarantee fairness, they do not guarantee that the lock will be
acquired by threads in the order that they called the lock().

In most case this works well, but in overload case the client
requests handling thread and _submit_thread could always successfully acquire the submit_mutex in a long time, which could make the MDLog::trim() get stuck. That means the MDS daemons will fill journal logs into the metadata pool, but couldn't trim the expired segments in time.

This will switch the submit_mutex to fair mutex and it could make sure that the all the submit_mutex waiters are in FIFO order and could get a change to be excuted in time.


Result:

All the waiters will be woke up one by one in FIFO mode.
Doc Type If docs needed, set a value Enhancement
Akash Raj 2023-07-14 10:58:33 UTC Doc Text Feature:

Switch the unfair Mutex lock to fair mutex.


Reason:

The implementations of the Mutex (e.g. std::mutex in C++) do not
guarantee fairness, they do not guarantee that the lock will be
acquired by threads in the order that they called the lock().

In most case this works well, but in overload case the client
requests handling thread and _submit_thread could always successfully acquire the submit_mutex in a long time, which could make the MDLog::trim() get stuck. That means the MDS daemons will fill journal logs into the metadata pool, but couldn't trim the expired segments in time.

This will switch the submit_mutex to fair mutex and it could make sure that the all the submit_mutex waiters are in FIFO order and could get a change to be excuted in time.


Result:

All the waiters will be woke up one by one in FIFO mode.
.Switch the unfair Mutex lock to fair mutex

Previously, the implementations of the _Mutex_, for example, `std::mutex` in _C++_, would not
guarantee fairness and would not guarantee that the lock would be acquired by threads in the order called `lock()`. In most cases, this worked well but in an overloaded case, the client
requests handling thread and _submit_ thread would always successfully acquire the _submit_mutex_ in a long time, causing `MDLog::trim()` to get stuck. That meant the MDS daemons would fill journal logs into the metadata pool, but could not trim the expired segments in time.

With this enhancement, the unfair Mutex lock is switched to fair mutex and all the _submit_mutex_ waiters are woken up one by one in FIFO mode.
errata-xmlrpc 2023-08-03 16:32:17 UTC Status VERIFIED RELEASE_PENDING
errata-xmlrpc 2023-08-03 16:45:09 UTC Resolution --- ERRATA
Status RELEASE_PENDING CLOSED
Last Closed 2023-08-03 16:45:09 UTC
errata-xmlrpc 2023-08-03 16:46:06 UTC Link ID Red Hat Product Errata RHBA-2023:4473

Back to bug 2158304