Bug 2292372 - [Prometheus mgr module][Backport-7.1] - Prometheus module is crashing during startup
Summary: [Prometheus mgr module][Backport-7.1] - Prometheus module is crashing during ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat Storage
Component: Ceph-Mgr Plugins
Version: 7.1
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ---
: 7.1z1
Assignee: Juan Miguel Olmo
QA Contact: Sayalee
ceph-docs@redhat.com
URL:
Whiteboard:
Depends On:
Blocks: 2274165 2303151 2303698
TreeView+ depends on / blocked
 
Reported: 2024-06-14 07:53 UTC by Juan Miguel Olmo
Modified: 2024-08-08 14:57 UTC (History)
5 users (show)

Fixed In Version: ceph-18.2.1-220
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2303151 (view as bug list)
Environment:
Last Closed: 2024-08-07 11:21:01 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHCEPH-9171 0 None None None 2024-06-14 07:55:10 UTC
Red Hat Product Errata RHBA-2024:5080 0 None None None 2024-08-07 11:21:06 UTC

Internal Links: 2274165

Description Juan Miguel Olmo 2024-06-14 07:53:25 UTC
backport mgr/prometheus: fix orch check to prevent Prometheus crash 

Upstream bug https://tracker.ceph.com/issues/63992
Upstream bug for Reef backport: https://tracker.ceph.com/issues/64339


```

debug 2024-01-09T18:31:05.181+0000 7ff5cc6ef700  1 mgr load Constructed class from module: progress
debug 2024-01-09T18:31:05.182+0000 7ff5cc6ef700  0 [prometheus DEBUG root] setting log level based on debug_mgr: INFO (2/5)
debug 2024-01-09T18:31:05.192+0000 7ff5cc6ef700 -1 no module 'rook'
debug 2024-01-09T18:31:05.192+0000 7ff5cc6ef700 -1 no module 'rook'
debug 2024-01-09T18:31:05.192+0000 7ff5cc6ef700 -1 mgr load Failed to construct class in 'prometheus'
debug 2024-01-09T18:31:05.193+0000 7ff66ef04700  0 [progress INFO root] Loading...
debug 2024-01-09T18:31:05.201+0000 7ff66ef04700  0 [progress INFO root] Loaded [<progress.module.GhostEvent object at 0x7ff742939080>, <progress.module.GhostEvent object at 0x7ff7429390b8>, <progress.module.GhostEvent object at 0x7ff7429390f0>, <progress.module.GhostEvent object at 0x7ff742939128>, <progress.module.GhostEvent object at 0x7ff742939160>, <progress.module.GhostEvent object at 0x7ff742939198>, <progress.module.GhostEvent object at 0x7ff7429391d0>, <progress.module.GhostEvent object at 0x7ff742939208>, <progress.module.GhostEvent object at 0x7ff742939240>, <progress.module.GhostEvent object at 0x7ff742939278>, <progress.module.GhostEvent object at 0x7ff7429392b0>, <progress.module.GhostEvent object at 0x7ff7429392e8>, <progress.module.GhostEvent object at 0x7ff742939320>, <progress.module.GhostEvent object at 0x7ff742939358>, <progress.module.GhostEvent object at 0x7ff742939390>, <progress.module.GhostEvent object at 0x7ff7429393c8>, <progress.module.GhostEvent object at 0x7ff742939400>, <progress.module.GhostEvent object at 0x7ff742939438>, <progress.module.GhostEvent object at 0x7ff742939470>, <progress.module.GhostEvent object at 0x7ff7429394a8>, <progress.module.GhostEvent object at 0x7ff7429394e0>, <progress.module.GhostEvent object at 0x7ff742939518>, <progress.module.GhostEvent object at 0x7ff742939550>, <progress.module.GhostEvent object at 0x7ff742939588>, <progress.module.GhostEvent object at 0x7ff7429395c0>, <progress.module.GhostEvent object at 0x7ff7429395f8>, <progress.module.GhostEvent object at 0x7ff742939630>, <progress.module.GhostEvent object at 0x7ff742939668>, <progress.module.GhostEvent object at 0x7ff7429396a0>, <progress.module.GhostEvent object at 0x7ff7429396d8>] historic events
debug 2024-01-09T18:31:05.202+0000 7ff66ef04700  0 [progress INFO root] Loaded OSDMap, ready.
debug 2024-01-09T18:31:05.203+0000 7ff67ff06700  0 [pg_autoscaler INFO root] _maybe_adjust
debug 2024-01-09T18:31:05.192+0000 7ff5cc6ef700 -1 mgr load Traceback (most recent call last):
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 1657, in _oremote
    return mgr.remote(o, meth, *args, **kwargs)
  File "/usr/share/ceph/mgr/mgr_module.py", line 2228, in remote
    args, kwargs)
ImportError: Module not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/ceph/mgr/prometheus/module.py", line 649, in __init__
    self.modify_instance_id = self.get_orch_status() and self.get_module_option(
  File "/usr/share/ceph/mgr/prometheus/module.py", line 869, in get_orch_status
    return self.available()[0]
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 1586, in inner
    completion = self._oremote(method_name, args, kwargs)
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 1661, in _oremote
    f_set = self.get_feature_set()
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 1586, in inner
    completion = self._oremote(method_name, args, kwargs)
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 1657, in _oremote
    return mgr.remote(o, meth, *args, **kwargs)
  File "/usr/share/ceph/mgr/mgr_module.py", line 2228, in remote
    args, kwargs)
ImportError: Module not found


```

Comment 12 errata-xmlrpc 2024-08-07 11:21:01 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat Ceph Storage 7.1 security and bug fix update.), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2024:5080


Note You need to log in before you can comment on or make changes to this bug.