Bug 1519705

Summary: [3.6.1] No document under .all index for non-ops kibana
Product: OpenShift Container Platform Reporter: Anping Li <anli>
Component: LoggingAssignee: Jeff Cantrill <jcantril>
Status: CLOSED ERRATA QA Contact: Junqi Zhao <juzhao>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.6.1CC: anli, aos-bugs, jcantril, juzhao, rmeggins
Target Milestone: ---Keywords: Regression
Target Release: 3.6.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: Trying to alias an index that does not exist Consequence: Bulk alias operation fails. Fix: Only alias .operations if there is at least one .operations index Result: .all will be an alias for all indices
 Story Points: ---
Clone Of: 1519694
: 1526287 (view as bug list) Environment:
Last Closed: 2018-01-23 17:59:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1519694    
Bug Blocks: 1526287    
Attachments:
Description Flags
enabled ops cluster,kibana UI, there are logs under .all index none

Description Anping Li 2017-12-01 09:04:27 UTC 
Description of problem:

No document under .all index for non-ops kibana when ops is enabled.


Version-Release number of selected component (if applicable):
logging-elasticsearch/images/v3.6.173.0.63-12
logging-elasticsearch/images/v3.6.173.0.63-12
logging-auth-proxy/images/v3.6.173.0.63-11
logging-fluentd/images/v3.6.173.0.63-11


How reproducible:
always

Steps to Reproduce:
1.  Deploy logging and enable ops elasticsearch

openshift_logging_install_logging=true
openshift_logging_image_prefix=registry.example.com/openshift3/
openshift_logging_namespace=logging
openshift_logging_es_cluster_size=1
openshift_logging_es_memory_limit=1Gi

openshift_logging_use_ops=true
openshift_logging_es_ops_memory_limit=1Gi
openshift_logging_es_ops_cluster_size=1


openshift_logging_use_mux=true
openshift_logging_use_mux_client=true




2. cluster-admin search records in non-ops kibana


Actual results:
No records in .all index 

Expected results:
.all index should include records from all projects

Additional info:
Comment 1 Jeff Cantrill 2017-12-04 21:30:22 UTC 
This occurs because there is no .operation index and we artificially add '.operations' to the projects to be aliased.  The bulk alias then fails because the index does not exist.  We will need to fix by checking for any indices and only adding .operations if we have a matching index.
Comment 2 Jeff Cantrill 2017-12-04 22:09:13 UTC 
Sample error message from logs:

[2017-12-04 21:02:58,033][ERROR][io.fabric8.elasticsearch.plugin.kibana.KibanaSeed] Error executing Alias request
[.operations.*] IndexNotFoundException[no such index]
Comment 3 openshift-github-bot 2017-12-11 23:06:43 UTC 
Commits pushed to master at https://github.com/openshift/origin-aggregated-logging

https://github.com/openshift/origin-aggregated-logging/commit/b2af1d20ec47ff9e9cd0eff846697d5214b5ca67
bug 1519705. bump openshift-elasticsearch-plugin-2.4.4.20 and build if necessary

https://github.com/openshift/origin-aggregated-logging/commit/4c5ca915194478a393b78b383464789d52804e9f
Merge pull request #835 from jcantrill/bump_244_20

Automatic merge from submit-queue.

bug 1519705. bump openshift-elasticsearch-plugin-2.4.4.20
Comment 5 Junqi Zhao 2017-12-18 01:08:28 UTC 
Enabled ops cluster, there are logs under .all index for non-ops kibana, see the attached picture.

Images:
logging-kibana/images/v3.6.173.0.89-1
logging-elasticsearch/images/v3.6.173.0.89-1
logging-fluentd/images/v3.6.173.0.89-1
logging-auth-proxy/images/v3.6.173.0.89-1
logging-curator/images/v3.6.173.0.89-1

elasticsearch-plugin in logging-elasticsearch image is openshift-elasticsearch-plugin-2.4.4.20 now
bash-4.2$ rpm -qa | grep elasticsearch-plugin
openshift-elasticsearch-plugin-2.4.4.20__redhat_1-1.el7.noarch
Comment 6 Junqi Zhao 2017-12-18 01:09:21 UTC 
Created attachment 1369217 [details]
enabled ops cluster,kibana UI, there are logs under .all index
Comment 9 errata-xmlrpc 2018-01-23 17:59:00 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0113