Bug 1979520

Summary: KeyError in /usr/share/ceph/mgr/stats/fs/perf_stats.py", line 129 every few milliseconds
Product: [Red Hat Storage] Red Hat Ceph Storage Reporter: Sebastian Wagner <sewagner>
Component: CephFSAssignee: Jos Collin <jcollin>
Status: CLOSED ERRATA QA Contact: Yogesh Mane <ymane>
Severity: low Docs Contact: Mary Frances Hull <mhull>
Priority: high    
Version: 5.0CC: agunn, ceph-eng-bugs, jcollin, pdonnell, sweil, tserlin, vereddy
Target Milestone: ---   
Target Release: 5.0z1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ceph-16.2.0-119.el8cp Doc Type: Bug Fix
Doc Text:
.Fixes KeyError appearing every few milliseconds in the MGR log Previously, `KeyError` was logged to the Ceph Manager log every few milliseconds. This was due to an attempt to remove an element from `client_metadata[in_progress]` dictionary with a non-existent key, resulting in a `KeyError`. As a result, locating other stack traces in the logs was difficult. This release fixes the code logic in the Ceph File System performance metrics and `KeyError` messages in the Ceph Manager log.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2021-11-02 16:38:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1959686    

Description Sebastian Wagner 2021-07-06 09:43:09 UTC 
The attached log files of https://bugzilla.redhat.com/show_bug.cgi?id=1978102 show 

debug 2021-06-30T06:09:02.171+0000 7f2dc1e85700 -1 mgr notify stats.notify:
debug 2021-06-30T06:09:03.228+0000 7f2dc1e85700 -1 mgr notify Traceback (most recent call last):
  File "/usr/share/ceph/mgr/stats/module.py", line 31, in notify
    self.fs_perf_stats.notify(notify_id)
  File "/usr/share/ceph/mgr/stats/fs/perf_stats.py", line 129, in notify
    result = self.client_metadata['in_progress'].pop(cmdtag)
KeyError: ''

Every few milliseconds in the MGR log. If this is indeed a bug, it would be great to have this backpacked as it makes finding other Tracebacks in the MGR log file somewhat irritating.
Comment 1 RHEL Program Management 2021-07-06 09:43:17 UTC 
Please specify the severity of this bug. Severity is defined here:
https://bugzilla.redhat.com/page.cgi?id=fields.html#bug_severity.
Comment 2 Jos Collin 2021-08-05 09:24:20 UTC 
The fix is under review:
https://github.com/ceph/ceph/pull/42608
Comment 14 errata-xmlrpc 2021-11-02 16:38:27 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat Ceph Storage 5.0 Bug Fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:4105