Description of problem:
When there are a large number of messages sent to the cluster log, such as slow ops warnings, the monitors can grow their db and fill up their disk. This occurs when the ingest rate of log messages is greater than the paxos_service_trim_max, which defaults to 500 entries - at this point logs are stored faster than they are deleted, and the store grows continuously.
Version-Release number of selected component (if applicable):
Any
How reproducible:
Always
Steps to Reproduce:
1. Set paxos_service_trim_min = 10 and paxos_service_trim_max = 100
2. Generate slow ops on a cluster by running I/O and setting the slow ops threshold very low (e.g. osd_op_complaint_time = 0.001).
Actual results:
mon db grows continuously
Expected results:
mon db should not grow continuously
Additional info:
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory (Red Hat Ceph Storage 5.0 bug fix and enhancement), and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHBA-2021:3294