Bug 1490719

Summary:

Enabled ops cluser,log in kibana-ops UI, there is no log entry under .all index, log entries only could be shown under .operations.* index

Product:

OpenShift Container Platform

Reporter:

Junqi Zhao <juzhao>

Component:

Logging

Assignee:

Jeff Cantrill <jcantril>

Status:

CLOSED ERRATA

QA Contact:

Junqi Zhao <juzhao>

Severity:

medium

Docs Contact:

Priority:

medium

Version:

3.6.1

CC:

anli, aos-bugs, jcantril, juzhao, rmeggins, xtian

Target Milestone:

---

Keywords:

Regression

Target Release:

3.6.z

Hardware:

Unspecified

OS:

Unspecified

Whiteboard:

Fixed In Version:

Doc Type:

No Doc Update

Doc Text:

undefined

Story Points:

---

Clone Of:

Environment:

Last Closed:

2017-12-07 07:11:23 UTC

Type:

Bug

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Bug Depends On:

1505683, 1506099

Bug Blocks:

Attachments:

Description	Flags
no log entries under .all index	none
log entries only could be found under .operatinns.* index	none
kibana UI, display .operations.* result	none
logging environment dump	none
kibana UI, it is noraml with logging-elasticsearch:v3.6.173.0.45-2	none
kibana ops UI, no log entries under .all index	none
logging dump output, logging-elasticsearch is v3.6.173.0.45-2	none
kibana-ops UI, there are logs under .all index	none

Description Junqi Zhao 2017-09-12 06:31:20 UTC

Created attachment 1324714 [details]
no log entries under .all index

Description of problem:
Deployed logging 3.6 with enabled ops cluser, log in kibana-ops UI,there is no log entry under .all index, log entries only could be shown under .operations.* index

Version-Release number of selected component (if applicable):
Images
logging-fluentd:v3.6.173.0.32-2
logging-kibana:v3.6.173.0.32-1
logging-elasticsearch:v3.6.173.0.32-1
logging-auth-proxy:v3.6.173.0.32-1
logging-curator:v3.6.173.0.32-1

How reproducible:
Always

Steps to Reproduce:
1. Deployed logging 3.6 with enabled ops cluser
2. Login kibana ops UI
3.

Actual results:
there is no log entry under .all index, log entries only could be shown under .operations.* index

Expected results:
there should be log entry under .all index

Additional info:
[OSEv3:children]
masters

[masters]
${MASTER_URL} openshift_public_hostname=${MASTER_URL}

[OSEv3:vars]
ansible_ssh_user=root
ansible_ssh_private_key_file="~/libra.pem"
deployment_type=openshift-enterprise

# Logging
openshift_logging_install_logging=true
openshift_logging_kibana_hostname=kibana.${SUB_DOMAIN}
openshift_logging_kibana_ops_hostname=kibana-ops.${SUB_DOMAIN}
public_master_url=https://${MASTER_URL}:${PORT}
openshift_logging_image_prefix=${IMAGE_PREFIX}
openshift_logging_image_version=v3.6
openshift_logging_namespace=logging
openshift_logging_use_ops=true
openshift_logging_elasticsearch_cpu_limit=500m
openshift_logging_elasticsearch_memory_limit=1Gi
openshift_logging_es_ops_cpu_limit=300m
openshift_logging_es_ops_memory_limit=512Mi

Comment 1 Junqi Zhao 2017-09-12 06:32:17 UTC

Created attachment 1324715 [details]
log entries only could be found under .operatinns.* index

Comment 2 Junqi Zhao 2017-09-12 06:33:36 UTC

Time range selected "Today"

Comment 5 Junqi Zhao 2017-10-12 02:31:24 UTC

Same error with https://bugzilla.redhat.com/show_bug.cgi?id=1499762

Comment 7 Junqi Zhao 2017-10-17 11:28:38 UTC

ops enabled cluster, there are no .all index in kibana-ops and kibana UI, for kibana-ops, it is ok, there is no error in the UI, but for kibana UI, when logging in kibana UI, the default kibana pages are all diplayed .operations.* results, this is wrong, and there is exception in UI: Discover: [exception] The index returned an empty result. You can use the Time Picker to change the time filter or select a higher time interval. See the attached pictures.

I think we should add .all index back for ops user.

Comment 8 Junqi Zhao 2017-10-17 11:29:45 UTC

Created attachment 1339668 [details]
kibana UI, display .operations.* result

Comment 9 Junqi Zhao 2017-10-17 11:36:57 UTC

Created attachment 1339669 [details]
logging environment dump

Comment 15 Junqi Zhao 2017-10-18 11:44:17 UTC

Created attachment 1340134 [details]
kibana UI, it is noraml with logging-elasticsearch:v3.6.173.0.45-2

Comment 16 Junqi Zhao 2017-10-18 11:45:52 UTC

Created attachment 1340135 [details]
kibana ops UI, no log entries under .all index

Comment 17 Junqi Zhao 2017-10-18 11:47:54 UTC

Created attachment 1340136 [details]
logging dump output, logging-elasticsearch is  v3.6.173.0.45-2

Comment 20 Junqi Zhao 2017-10-25 02:43:47 UTC

Blocks by https://bugzilla.redhat.com/show_bug.cgi?id=1505683

Comment 21 Anping Li 2017-10-30 06:21:02 UTC

Test with logging: v3.6.173.0.62. operations.* index appears in regurlar kibana, and Exception [1] appears.  

Only the .operations.* index is in kibana-ops.  I remember that  there should be  an all item. When and why we drop the all item?

[1] Exception in regurlar kibana.
Error: [exception] The index returned an empty result. You can use the Time Picker to change the time filter or select a higher time interval
ErrorAbstract@https://kibana.apps.1030-knj.qe.rhcloud.com/bundles/commons.bundle.js?v=10229:95501:20
StatusCodeError@https://kibana.apps.1030-knj.qe.rhcloud.com/bundles/commons.bundle.js?v=10229:95663:6
respond@https://kibana.apps.1030-knj.qe.rhcloud.com/bundles/commons.bundle.js?v=10229:96868:16
checkRespForFailure@https://kibana.apps.1030-knj.qe.rhcloud.com/bundles/commons.bundle.js?v=10229:96831:8
[24]</AngularConnector.prototype.request/<@https://kibana.apps.1030-knj.qe.rhcloud.com/bundles/commons.bundle.js?v=10229:95449:8
processQueue@https://kibana.apps.1030-knj.qe.rhcloud.com/bundles/commons.bundle.js?v=10229:42452:29
scheduleProcessQueue/<@https://kibana.apps.1030-knj.qe.rhcloud.com/bundles/commons.bundle.js?v=10229:42468:28
$RootScopeProvider/this.$get</Scope.prototype.$eval@https://kibana.apps.1030-knj.qe.rhcloud.com/bundles/commons.bundle.js?v=10229:43696:17
$RootScopeProvider/this.$get</Scope.prototype.$digest@https://kibana.apps.1030-knj.qe.rhcloud.com/bundles/commons.bundle.js?v=10229:43507:16
$RootScopeProvider/this.$get</Scope.prototype.$apply@https://kibana.apps.1030-knj.qe.rhcloud.com/bundles/commons.bundle.js?v=10229:43804:14
done@https://kibana.apps.1030-knj.qe.rhcloud.com/bundles/commons.bundle.js?v=10229:38253:37
completeRequest@https://kibana.apps.1030-knj.qe.rhcloud.com/bundles/commons.bundle.js?v=10229:38451:8
requestLoaded@https://kibana.apps.1030-knj.qe.rhcloud.com/bundles/commons.bundle.js?v=10229:38392:10

Comment 22 Anping Li 2017-11-16 10:48:37 UTC

There is .all index in v3.6.173.0.63. but I got message "No results found"  when click .all index.

Comment 23 Jeff Cantrill 2017-11-16 13:33:59 UTC

@Anping,

* Can you please query ES directly to confirm you have data? Dumping the indices will show you a document count
* Do you have a time range that includes data?

Comment 25 Junqi Zhao 2017-11-22 09:01:55 UTC

Tested with v3.6.173.0.78-1 logging images, deployed logging with enabled ops cluster, There are logs under .all index in kibana-ops UI. See the snapshot

Please set this defect as ON_QA, so we can close it.

Comment 26 Junqi Zhao 2017-11-22 09:02:31 UTC

Created attachment 1357314 [details]
kibana-ops UI, there are logs under .all index

Comment 27 Junqi Zhao 2017-11-23 00:31:32 UTC

Set this defect to VERIFIED as per Comment 25

Comment 30 errata-xmlrpc 2017-12-07 07:11:23 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:3389