Back to bug 1548104

Who When What Removed Added
Mike Fiedler 2018-02-22 17:59:46 UTC Whiteboard aos-scalability-39
Rich Megginson 2018-02-22 18:28:10 UTC CC lvlcek, pportant
Jeff Cantrill 2018-02-23 15:40:41 UTC Status NEW POST
Target Release --- 3.9.0
Link ID Github openshift/origin-aggregated-logging/pull/974
Jeff Cantrill 2018-02-26 21:39:12 UTC Link ID Github openshift/openshift-ansible/pull/7294
Jeff Cantrill 2018-02-28 20:13:32 UTC Link ID Github openshift/openshift-ansible/pull/7294
Jeff Cantrill 2018-03-07 16:00:41 UTC Status POST MODIFIED
errata-xmlrpc 2018-03-08 15:20:24 UTC Status MODIFIED ON_QA
Jeff Cantrill 2018-03-09 13:43:56 UTC Doc Text Cause: Fluentd inserts code into Elasticsearch using bulk inserts but relies upon Elasticsearch to generate UUIDs for each document. It does not remove successfully indexed documents from the bulk payload when the bulk operation fails.

Consequence: The initial payload is resubmitted and documents that were successfully indexed are submitted again which will result in duplicate documents with different uuids.

Fix: Generate document ids before submitting bulk insert requests.

Result: Elasticsearch will disregard insert of documents that already exist in the data store and insert documents that do not.
Doc Type If docs needed, set a value Bug Fix
Peter Portante 2018-03-10 19:57:12 UTC Doc Text Cause: Fluentd inserts code into Elasticsearch using bulk inserts but relies upon Elasticsearch to generate UUIDs for each document. It does not remove successfully indexed documents from the bulk payload when the bulk operation fails.

Consequence: The initial payload is resubmitted and documents that were successfully indexed are submitted again which will result in duplicate documents with different uuids.

Fix: Generate document ids before submitting bulk insert requests.

Result: Elasticsearch will disregard insert of documents that already exist in the data store and insert documents that do not.
Cause: Fluentd inserts documents (logs) into Elasticsearch using the bulk insert API but relies upon Elasticsearch to generate UUIDs for each document. It does not remove successfully indexed documents from the bulk payload when the bulk operation fails.

Consequence: The initial payload is resubmitted and documents that were successfully indexed are submitted again which will result in duplicate documents with different uuids.

Fix: Generate document ids before submitting bulk insert requests.

Result: Elasticsearch will disregard insert of documents that already exist in the data store and insert documents that do not.
Junqi Zhao 2018-03-12 02:31:15 UTC CC juzhao
QA Contact anli mifiedle
Mike Fiedler 2018-03-12 22:41:53 UTC Status ON_QA VERIFIED
Jeff Cantrill 2018-03-15 13:10:15 UTC Blocks 1556896
Jeff Cantrill 2018-03-15 13:12:47 UTC Blocks 1556897
errata-xmlrpc 2018-03-28 00:46:35 UTC Status VERIFIED RELEASE_PENDING
errata-xmlrpc 2018-03-28 14:30:19 UTC Status RELEASE_PENDING CLOSED
Resolution --- ERRATA
Last Closed 2018-03-28 10:30:19 UTC
errata-xmlrpc 2018-03-28 14:30:52 UTC Link ID Red Hat Product Errata RHBA-2018:0489

Back to bug 1548104