Bug 1857826 - Index Management underestimate number of documents in the index to rollover
Summary: Index Management underestimate number of documents in the index to rollover
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 4.6
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 4.6.0
Assignee: Lukas Vlcek
QA Contact: Qiaoling Tang
Depends On:
Blocks: 1860135 1861009
TreeView+ depends on / blocked
Reported: 2020-07-16 16:01 UTC by Lukas Vlcek
Modified: 2020-10-27 16:15 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1860135 (view as bug list)
Last Closed: 2020-10-27 16:15:14 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Github openshift elasticsearch-operator pull 425 0 None closed Bug 1857826: Assume average log message is 1K 2020-10-26 13:45:36 UTC
Red Hat Product Errata RHBA-2020:4196 0 None None None 2020-10-27 16:15:45 UTC

Description Lukas Vlcek 2020-07-16 16:01:13 UTC
Description of problem:

The IndexManagement is configured to rollover index if any of existing conditions is met. We have conditions on index size, number of document and age. Currently the condition on number of documents in the index was calculated assuming the size of individual document (a single log) is up to 1M. This is oversized for most of the real cases and is causing the index to be rolled over when 120.000 documents are stored in the index (given 3/1 prim/replicas sharding). This can easily lead to 10 new infra indices per hour on an "idle" OCP cluster.

We are going to update the condition to assume average size of individual log is a lot less (~1K). 

Version-Release number of selected component (if applicable):

All 4.x existing releases that has a new data model (that is 4.5.x and 4.6).

Comment 4 Qiaoling Tang 2020-07-24 02:24:06 UTC
Verified with ose-elasticsearch-operator-v4.6.0-202007231721.p0

Comment 6 errata-xmlrpc 2020-10-27 16:15:14 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.