Bug 1548038

Summary: .all alias not being updated with new indices after a certain date
Product: OpenShift Container Platform Reporter: David Caldwell <dcaldwel>
Component: LoggingAssignee: Lukas Vlcek <lvlcek>
Status: CLOSED ERRATA QA Contact: Anping Li <anli>
Severity: high Docs Contact:
Priority: medium    
Version: 3.6.1CC: agerami, anli, aos-bugs, cstark, farandac, fshaikh, jcantril, jmalde, lvlcek, per.carlson, qitang, rkant, rmeggins, sgaikwad, smunilla
Target Milestone: ---   
Target Release: 3.7.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: Aliasing only occurs when a user logs into Kibana. If the the openshift-elasticsearch-plugin is not functioning correctly then aliasing may not occur. Consequence: .all alias will not be associated with all indices Fix: Use Elasticsearch index templates to alias all newly created indices Result: All newly created indices will be aliased to .all
Story Points: ---
Clone Of:
: 1548463 1577877 (view as bug list) Environment:
Last Closed: 2018-07-24 04:28:13 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1548463, 1577877    
Attachments:
Description Flags
list of updates applied none

Description David Caldwell 2018-02-22 15:03:59 UTC
Created attachment 1399435 [details]
list of updates applied

Description of problem:
The .all alias exists but after a certain date, it is longer updated - even though other indices _are_ created and are searchable after this date. This date coincides with platform updates the customer applied (see attached for a list) and then a reboot.

OCP was a fresh install and not an upgrade.

Version-Release number of selected component (if applicable):
OCP 3.6.173.0.5

Dockerfile info: /root/buildinfo/Dockerfile-openshift3-logging-curator-v3.6.173.0.96-2
"build-date"="2018-01-10T22:14:06.235392"

Dockerfile info: /root/buildinfo/Dockerfile-openshift3-logging-elasticsearch-v3.6.173.0.96-3
"build-date"="2018-01-18T13:02:43.616483"

Dockerfile info: /root/buildinfo/Dockerfile-openshift3-logging-auth-proxy-v3.6.173.0.96-2
"build-date"="2018-01-10T22:14:09.238369"

Dockerfile info: /root/buildinfo/Dockerfile-openshift3-logging-fluentd-v3.6.173.0.96-2
"build-date"="2018-01-10T22:14:59.119836"

How reproducible:
Not certain. The .all alias stopped being updated after the customer applied platform updates and rebooted (see attached for list of updates applied).

Steps to Reproduce:
1.install OCP 3.6.173.0.5
2.apply updates listed in attached
3.reboot platform

Actual results:
.all alias stops updating

Expected results:
.all alias doesn't stop updating and continues to be searchable

Additional info:
Customer also has a 3.7 cluster that may be exhibiting the same problems.

logging-dump.sh script attached.

Comment 10 Jeff Cantrill 2018-03-05 15:51:21 UTC
No. This will alias the indices and make no modification to the data.  The customer will need to manually alias old indices

Comment 22 Jeff Cantrill 2018-04-24 21:25:25 UTC
Can you advise if these indices were created before or after the deployment of the new images?  The change exists [1] in the referenced version but will only apply to new indices

http://pkgs.devel.redhat.com/cgit/rpms/logging-elasticsearch-docker/tree/index_templates/com.redhat.viaq-openshift-operations.template.json?h=rhaos-3.7-rhel-7

Comment 23 Qiaoling Tang 2018-04-25 02:39:05 UTC
These indices were created after the deployment of the new images. The test env was a new env and I didn't use dynamic pv.

Comment 32 Jeff Cantrill 2018-07-05 16:12:30 UTC
Suresh, we will not be backporting to 3.5.  Consider upgrading to a newer release.  Alternatively you could modify the index-templates to add the alias upon each index creation https://github.com/openshift/origin-aggregated-logging/pull/1230

Comment 33 Anping Li 2018-07-13 02:13:48 UTC
The .all alias was added  when index is created in v3.7.57-4.

Comment 35 errata-xmlrpc 2018-07-24 04:28:13 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:2234