Bug 1513284

Summary: [3.5.1]project indices can not be found on kibna UI
Product: OpenShift Container Platform Reporter: Junqi Zhao <juzhao>
Component: LoggingAssignee: Jeff Cantrill <jcantril>
Status: CLOSED ERRATA QA Contact: Junqi Zhao <juzhao>
Severity: high Docs Contact:
Priority: high    
Version: 3.5.1CC: anli, aos-bugs, juzhao, rmeggins, wsun
Target Milestone: ---Keywords: Regression
Target Release: 3.5.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
undefined
 Story Points: ---
Clone Of: Environment:
Last Closed: 2018-04-12 05:59:07 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
enabled ops cluster, kibana UI, there is not result under .all index
none
disabled ops cluster, kibana UI, although project indices can not be found on kibna UI, project logs can be searched from kibna UI none

Description Junqi Zhao 2017-11-15 05:25:58 UTC 
Created attachment 1352385 [details]
enabled ops cluster, kibana UI, there is not result under .all index

Description of problem:
1. Deploy logging 3.5 without enabled ops cluster and create project, check project logs from kibana. Although project indices can not be found on kibna UI, project logs can be searched from kibna UI


2. Deploy logging 3.5 with enabled ops cluster and create project, check project logs from kibana. Project indices can not be found on kibna UI, and project logs can not be searched from kibna UI, there is not log under .all index,
operation logs can  be searched from kibna-ops UI

both 1 and 2 scenario, project indices are generated.
# oc exec ${es_pod} -- curl -s -k --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key https://logging-es:9200/project.*/_search?size=13\&sort=@timestamp:desc | python -mjson.tool | grep project
                "_index": "project.testing.6cc80ae5-c9ad-11e7-a6e2-fa163ef332b3.2017.11.15",
                "_index": "project.testing.6cc80ae5-c9ad-11e7-a6e2-fa163ef332b3.2017.11.15",
                "_index": "project.java.c737f288-c9a9-11e7-a6e2-fa163ef332b3.2017.11.15",
                "_index": "project.testing.6cc80ae5-c9ad-11e7-a6e2-fa163ef332b3.2017.11.15",
                "_index": "project.java.c737f288-c9a9-11e7-a6e2-fa163ef332b3.2017.11.15",
                "_index": "project.install-test.369e900a-c9a7-11e7-9ac3-fa163ef332b3.2017.11.15",
                "_index": "project.install-test.369e900a-c9a7-11e7-9ac3-fa163ef332b3.2017.11.15",
                "_index": "project.testing.6cc80ae5-c9ad-11e7-a6e2-fa163ef332b3.2017.11.15",
                "_index": "project.java.c737f288-c9a9-11e7-a6e2-fa163ef332b3.2017.11.15",
                "_index": "project.testing.6cc80ae5-c9ad-11e7-a6e2-fa163ef332b3.2017.11.15",
                "_index": "project.java.c737f288-c9a9-11e7-a6e2-fa163ef332b3.2017.11.15",
                "_index": "project.testing.6cc80ae5-c9ad-11e7-a6e2-fa163ef332b3.2017.11.15",
                "_index": "project.java.c737f288-c9a9-11e7-a6e2-fa163ef332b3.2017.11.15",


Version-Release number of selected component (if applicable):

images:
logging-elasticsearch/images/3.5.0-56
logging-kibana/images/3.5.0-51
logging-fluentd/images/3.5.0-46
logging-auth-proxy/images/3.5.0-45
logging-curator/images/v3.5.5.31.47-8

# openshift version
openshift v3.5.5.31.47
kubernetes v1.5.2+43a9be4
etcd 3.1.0

How reproducible:
Always

Steps to Reproduce:
1. Deploy logging 3.5 without enabled ops cluster and create project, check project logs from kibana
2. Deploy logging 3.5 with enabled ops cluster and create project, check project logs from kibana
3.

Actual results:
For step1:. although project indices can not be found on kibna UI, project logs can be searched from kibna UI
For step 2:  project indices can not be found on kibna UI, and project logs can not be searched from kibna UI

Expected results:
For step1:  project indices can be shown on kibna UI
For step 2:  project indices can be shown on kibna UI and project logs can be searched from kibna UI


Additional info:
Comment 1 Junqi Zhao 2017-11-15 05:27:54 UTC 
Created attachment 1352386 [details]
disabled ops cluster, kibana UI, although project indices can not be found on kibna UI, project logs can be searched from kibna UI
Comment 4 Junqi Zhao 2017-11-22 08:03:38 UTC 
workaround to show project indices
1. Login in kibana UI
2. Go to "settings" -> "indices", "Configure an index pattern" page, add "project.${project_name}*" in "Index name or pattern“ field, select @timestamp from "Time-field name" field. After that, navigate to "Discovery" tab
Comment 5 Anping Li 2017-11-30 10:33:42 UTC 
We still couldn't view the index by default when use the stage images
logging-elasticsearch/images/3.5.0-57
logging-auth-proxy/images/3.5.0-45
logging-kibana/images/3.5.0-51
logging-fluentd/images/3.5.0-46
Comment 6 Junqi Zhao 2017-12-01 08:24:52 UTC 
Tested with ops and non-ops cluster, project indices could be viewed from kibana UI. But we found another regression bug: https://bugzilla.redhat.com/show_bug.cgi?id=1519694

Images:
logging-elasticsearch-3.5.0-58
logging-auth-proxy-3.5.0-45
logging-kibana-3.5.0-51
logging-fluentd-3.5.0-46
logging-curator-v3.5.5.31.47-10

Please set this defect to ON_QA
Comment 7 Junqi Zhao 2018-03-22 08:21:29 UTC 
Enabled ops cluster, project indices can be found and there are logs under .all index on kibana UI


Images
logging-curator-v3.5.5.31.64-1
logging-elasticsearch-3.5.0-62
logging-fluentd-3.5.0-53
logging-kibana-3.5.0-54
logging-auth-proxy-3.5.0-47
Comment 8 Junqi Zhao 2018-03-22 08:22:16 UTC 
# openshift version
openshift v3.5.5.31.64
kubernetes v1.5.2+43a9be4
etcd 3.1.0
Comment 11 errata-xmlrpc 2018-04-12 05:59:07 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:1106