Bug 1857826

Summary: Index Management underestimate number of documents in the index to rollover
Product: OpenShift Container Platform Reporter: Lukas Vlcek <lvlcek>
Component: LoggingAssignee: Lukas Vlcek <lvlcek>
Status: CLOSED ERRATA QA Contact: Qiaoling Tang <qitang>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.6CC: aos-bugs, mburke
Target Milestone: ---   
Target Release: 4.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1860135 (view as bug list) Environment:
Last Closed: 2020-10-27 16:15:14 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1860135, 1861009    

Description Lukas Vlcek 2020-07-16 16:01:13 UTC 
Description of problem:

The IndexManagement is configured to rollover index if any of existing conditions is met. We have conditions on index size, number of document and age. Currently the condition on number of documents in the index was calculated assuming the size of individual document (a single log) is up to 1M. This is oversized for most of the real cases and is causing the index to be rolled over when 120.000 documents are stored in the index (given 3/1 prim/replicas sharding). This can easily lead to 10 new infra indices per hour on an "idle" OCP cluster.

We are going to update the condition to assume average size of individual log is a lot less (~1K). 

Version-Release number of selected component (if applicable):

All 4.x existing releases that has a new data model (that is 4.5.x and 4.6).
Comment 4 Qiaoling Tang 2020-07-24 02:24:06 UTC 
Verified with ose-elasticsearch-operator-v4.6.0-202007231721.p0
Comment 6 errata-xmlrpc 2020-10-27 16:15:14 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196