Bug 1553257 - [REF]-Use multiple primary shards for the .operations indices by default matching the number of data nodes in the cluster
Summary: [REF]-Use multiple primary shards for the .operations indices by default matc...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 3.9.0
Hardware: All
OS: Linux
high
low
Target Milestone: ---
: 3.9.z
Assignee: Jeff Cantrill
QA Contact: Anping Li
URL:
Whiteboard:
Depends On: 1552977
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-03-08 15:14 UTC by Jeff Cantrill
Modified: 2018-08-09 22:14 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Feature: Allow the number if indices and replicas to be configured using environment variables Reason: The logs collected for infra services consumes a large portion of the available disk space. Spreading the data across available nodes by modifying the replica and shard settings allow Elasticsearch to better support these large amounts of data Result: Improved performance in Elasticsearch when there are large amounts of data from infra services.
Clone Of: 1552977
Environment:
Last Closed: 2018-08-09 22:13:46 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift origin-aggregated-logging pull 1018 0 None None None 2018-03-15 12:30:33 UTC
Red Hat Product Errata RHBA-2018:2335 0 None None None 2018-08-09 22:14:25 UTC

Description Jeff Cantrill 2018-03-08 15:14:59 UTC 
+++ This bug was initially created as a clone of Bug #1552977 +++

Sharding the .operations indices (and .orphan indices) using a primary shard count that matches the number of nodes in the cluster will provide for an even distribution of disk usage and load across cluster members to help with high operational logging rates.  

This has little effect in low logging rate situations.

--- Additional comment from Peter Portante on 2018-03-07 20:07:30 EST ---

See the following gist for an example: https://gist.github.com/portante/f8cfecad1c6b69cdc1736ce464501d6f
Comment 1 Jeff Cantrill 2018-03-08 15:29:26 UTC 
Master change: https://github.com/openshift/origin-aggregated-logging/pull/1007
Comment 2 openshift-github-bot 2018-03-15 04:35:04 UTC 
Commit pushed to master at https://github.com/openshift/origin-aggregated-logging

https://github.com/openshift/origin-aggregated-logging/commit/57d0775bb0dc89b8a83692d8e63beed80c705163
bug 1553257. Shard operations and orphaned indices
Comment 3 Jeff Cantrill 2018-03-15 12:30:34 UTC 
https://github.com/openshift/origin-aggregated-logging/pull/1018
Comment 5 Anping Li 2018-04-17 04:45:24 UTC 
Seems the fix is not in openshift-ansible:v3.9.22 and logging-elasticsearch/images/v3.9.22-1.
Comment 8 Qiaoling Tang 2018-04-20 05:37:17 UTC 
The fix isn't in openshift v3.9.24 and logging-elasticsearch/images/v3.9.25-1.
Comment 12 Anping Li 2018-07-31 10:35:13 UTC 
Fix in 3.9.40
Comment 14 errata-xmlrpc 2018-08-09 22:13:46 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:2335
Note You need to log in before you can comment on or make changes to this bug.