Bug 2142143

Summary:	mon/Elector: notify_rank_removed erase rank from both live_pinging and dead_pinging sets for highest ranked MON
Product:	[Red Hat Storage] Red Hat Ceph Storage	Reporter:	Kamoltat (Junior) Sirivadhna <ksirivad>
Component:	RADOS	Assignee:	Kamoltat (Junior) Sirivadhna <ksirivad>
Status:	CLOSED ERRATA	QA Contact:	Pawan <pdhiran>
Severity:	medium	Docs Contact:	Eliska <ekristov>
Priority:	unspecified
Version:	5.0	CC:	akupczyk, amathuri, bhubbard, bkunal, ceph-eng-bugs, cephqe-warriors, choffman, ekristov, kdreyer, ksirivad, lflores, nojha, pdhange, pdhiran, rfriedma, rzarzyns, sseshasa, vereddy, vumrao
Target Milestone:	---	Keywords:	Rebase
Target Release:	6.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	ceph-17.2.5-2.el9cp	Doc Type:	Bug Fix
Doc Text:	.The targeted `rank_removed` no longer gets stuck in `live_pinging` and `dead_pinging` states Previously, in some cases, the `paxos_size` of the Monitor Map would get updated before the rank of the monitor was changed. For example, `paxos_size` would get reduced from 5 to 4, but the highest rank of the Monitors was still 4, thus the old code would skip deleting the rank from `dead_pinging` state. This would cause the targeted rank to remain in `dead_pinging` forever, which would then cause strange `peer_tracker` scores in election strategy: 3. With this fix, a case is added when `rank_removed == paxos_size()` that erases the targeted `rank_removed` from both the `live_pinging` and `dead_pinging` states and the rank does not get stuck forever in either of these sets.	Story Points:	---
Clone Of:
Clones:	2142174 (view as bug list)		Environment:
Last Closed:	2023-03-20 18:59:13 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2126050, 2142174

Description Kamoltat (Junior) Sirivadhna 2022-11-11 19:24:48 UTC

Description of problem:

Added a case where we are removing the highest rank monitor
in notify_rank_removed, the old version did not deal with this
since it would only go into the loop when rank_removed < paxos_size().
Therefore, we added an else case for when rank_removed == paxos_size(),
we erase the rank from both live_pinging and dead_pinging set.

Comment 1 Kamoltat (Junior) Sirivadhna 2022-11-15 17:23:52 UTC

The patch is already in 6.0 as part of reabase ... Moving to POST

Comment 8 Kamoltat (Junior) Sirivadhna 2022-11-29 03:50:34 UTC

modified docs LGTM!

Comment 10 Kamoltat (Junior) Sirivadhna 2022-12-14 19:43:58 UTC

Hi Pawan,

This fix doesn't address the issue you are showing, especially if there is a known issue https://bugzilla.redhat.com/show_bug.cgi?id=2151501. Therefore, if there is no other issue present on this, then I think it should be fine. The motivation behind this PR is correctly removing stuff in  live_pinging and dead_pinging sets. The messed up score is probably part of https://bugzilla.redhat.com/show_bug.cgi?id=2151501

Comment 11 Ken Dreyer (Red Hat) 2022-12-14 21:46:59 UTC

Kamoltat, what is the next action you expect from Pawan?

Comment 33 errata-xmlrpc 2023-03-20 18:59:13 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat Ceph Storage 6.0 Bug Fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:1360