Bug 1647624

Summary:	Ceph msgr should log when it reaches the DispatchQueue throttle limit
Product:	[Red Hat Storage] Red Hat Ceph Storage	Reporter:	Patrick Donnelly <pdonnell>
Component:	RADOS	Assignee:	Brad Hubbard <bhubbard>
Status:	CLOSED ERRATA	QA Contact:	skanta
Severity:	high	Docs Contact:	Eliska <ekristov>
Priority:	high
Version:	3.1	CC:	aivaraslaimikis, akraj, akupczyk, amathuri, bhubbard, ceph-eng-bugs, cephqe-warriors, choffman, ekristov, gfarnum, ksirivad, lflores, ngangadh, nojha, pasik, pdhange, rfriedma, rperiyas, rzarzyns, skanta, sseshasa, tserlin, vereddy, vumrao
Target Milestone:	---
Target Release:	6.0
Hardware:	All
OS:	All
Whiteboard:	DevNeeded
Fixed In Version:	ceph-17.2.3-18.el9cp	Doc Type:	Enhancement
Doc Text:	.Low-level log messages are introduced to warn user about hitting throttle limits Previously, there was a lack of low-level logging indication that throttle limits were hit, causing these occurrences to incorrectly have the appearance of a networking issue. With this release, the introduction of low-level log messages makes it much clearer that the throttle limits are hit.	Story Points:	---
Clone Of:
Clones:	2110008 (view as bug list)		Environment:
Last Closed:	2023-03-20 18:55:33 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2100747, 2110008, 2126050

Description Patrick Donnelly 2018-11-07 22:54:42 UTC

Description of problem:

Right now there is no low debug output indicating that the throttle is reached which, in production, can give the appearance of network issues. In particular, hitting the DispatchQueue::dispatch_throttler limit will even prevent fast dispatch of critical messages.

There should be a message whenever we hit the limit and a debug message has not been output recently (30 seconds?).


Version-Release number of selected component (if applicable):

3.0

Comment 8 Giridhar Ramaraju 2019-08-05 13:11:02 UTC

Updating the QA Contact to a Hemant. Hemant will be rerouting them to the appropriate QE Associate. 

Regards,
Giri

Comment 9 Giridhar Ramaraju 2019-08-05 13:12:04 UTC

Updating the QA Contact to a Hemant. Hemant will be rerouting them to the appropriate QE Associate. 

Regards,
Giri

Comment 10 Yaniv Kaul 2020-06-22 09:45:47 UTC

What's the status of this BZ?

Comment 11 Patrick Donnelly 2020-06-22 16:52:53 UTC

(In reply to Yaniv Kaul from comment #10)
> What's the status of this BZ?

No developer work on it yet. Changing that now...

Comment 18 Greg Farnum 2022-01-05 23:45:32 UTC

Moving this out to 5.2 — it's still pending review upstream and I'm not aware of any urgent need.

Patrick, this was originally your request — please prioritize review for it if you think it's important.

Comment 56 errata-xmlrpc 2023-03-20 18:55:33 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat Ceph Storage 6.0 Bug Fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:1360