Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1933978

Summary: Unable to access/use index pattern after a couple of days
Product: OpenShift Container Platform Reporter: Simon Reber <sreber>
Component: LoggingAssignee: ewolinet
Status: CLOSED ERRATA QA Contact: Anping Li <anli>
Severity: high Docs Contact:
Priority: high    
Version: 4.5CC: alchan, andcosta, anisal, anli, aos-bugs, cruhm, david.karlsen, dgautam, ewolinet, jcantril, kiyyappa, luaparicio, mfiedler, mrobson, naygupta, ocasalsa, openshift-bugs-escalate, pauwebst, periklis, prdeshpa, qitang, rpalathi, vsamanth
Target Milestone: ---   
Target Release: 4.6.z   
Hardware: x86_64   
OS: Linux   
Whiteboard: logging-exploration
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-05-12 12:16:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Simon Reber 2021-03-02 07:59:55 UTC
Description of problem:

After addressing https://bugzilla.redhat.com/show_bug.cgi?id=1895581 it was possible to create index pattern for all users. This is also working ever since but now there is a new problem.

After a couple of days when the index pattern was created, the user is again redirected (upon login) by Kibana to the page, to create an index pattern (even though it still should exist).

Interesting enough, in the Saved Objects overview the `app` index pattern is reported and visible. But it can't be viewed, modified or even removed. So the user is again pretty much blocked as Kibana does not allow to run activity as it does expect an index pattern being created but also has problem with the one that already exists.

Also users are reporting error messages such as "Saved object is missing" and "Could not locate that index-pattern".

Restarting Kibana does not help as the problem persists. The only work-around currently is to remove the user indices as afterwards, the user is again able to create an index pattern (this is working) and can also use Kibana normally (but again after several days the issue comes again).

Also generally the user indices look OK and I suspect that it's rather a problem with permissions that the user is unable to access the user indices or similar

Version-Release number of selected component (if applicable):

 - clusterlogging.4.5.0-202102032236.p0
 - elasticsearch-operator.4.5.0-202102041049.p0

How reproducible:

 - N/A

Steps to Reproduce:
1. Not yet been able to reproduce

Actual results:

Users are redirected to index pattern creation page and is unable to use Kibana as the index pattern does exist respectively can't be accessed for some reason

Expected results:

Kibana to work continuously without any issue

Additional info:

Comment 33 Anping Li 2021-03-31 12:02:11 UTC
By accident, I reproduced this issue with following steps. 
1) Deploy Cluster Logging 4.4.
2) Upgrade Logging to 4.5.36  --> All works well
3) Delete CLO operators and EO operators
4) Install operator 4.5.33.   --> Save

Comment 35 Anping Li 2021-03-31 14:00:38 UTC
# Kibana popup message below when click the index pattern.
Could not locate that index-pattern (id: 468eab30-920c-11eb-ba63-91f02039c02d), click here to re-create it 

#Delete the saved objects(Index pattern), report 403.

{"type":"response","@timestamp":"2021-03-31T13:48:26Z","tags":[],"pid":121,"method":"delete","statusCode":403,"req":{"url":"/api/saved_objects/index-pattern/468eab30-920c-11eb-ba63-91f02039c02d","method":"delete","headers":{"host":"kibana-openshift-logging.apps.anli461.qe.gcp.devcluster.openshift.com","user-agent":"Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:87.0) Gecko/20100101 Firefox/87.0","accept":"*/*","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.5","content-type":"application/json","forwarded":"for=61.51.136.151;host=kibana-openshift-logging.apps.anli461.qe.gcp.devcluster.openshift.com;proto=https","kbn-version":"6.8.1","origin":"https://kibana-openshift-logging.apps.anli461.qe.gcp.devcluster.openshift.com","referer":"https://kibana-openshift-logging.apps.anli461.qe.gcp.devcluster.openshift.com/app/kibana","x-forwarded-access-token":"sha256~FXL-wh_YXqNJe2_1DaE7zkMxd7l0NE-xgdgkb2u6fXA","x-forwarded-email":"testuser-0","x-forwarded-for":"61.51.136.151, 10.131.0.5","x-forwarded-host":"kibana-openshift-logging.apps.anli461.qe.gcp.devcluster.openshift.com","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-user":"testuser-0","securitytenant":"__user__"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana-openshift-logging.apps.anli461.qe.gcp.devcluster.openshift.com/app/kibana"},"res":{"statusCode":403,"responseTime":36,"contentLength":9},"message":"DELETE /api/saved_objects/index-pattern/468eab30-920c-11eb-ba63-91f02039c02d 403 36ms - 9.0B"}

#Create index pattern, report 403.

{"type":"response","@timestamp":"2021-03-31T13:56:55Z","tags":[],"pid":121,"method":"post","statusCode":403,"req":{"url":"/api/saved_objects/index-pattern?overwrite=false","method":"post","headers":{"host":"kibana-openshift-logging.apps.anli461.qe.gcp.devcluster.openshift.com","user-agent":"Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:87.0) Gecko/20100101 Firefox/87.0","content-length":"66","accept":"*/*","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.5","content-type":"application/json","forwarded":"for=61.51.136.151;host=kibana-openshift-logging.apps.anli461.qe.gcp.devcluster.openshift.com;proto=https","kbn-version":"6.8.1","origin":"https://kibana-openshift-logging.apps.anli461.qe.gcp.devcluster.openshift.com","referer":"https://kibana-openshift-logging.apps.anli461.qe.gcp.devcluster.openshift.com/app/kibana","x-forwarded-access-token":"sha256~FXL-wh_YXqNJe2_1DaE7zkMxd7l0NE-xgdgkb2u6fXA","x-forwarded-email":"testuser-0","x-forwarded-for":"61.51.136.151, 10.128.2.10","x-forwarded-host":"kibana-openshift-logging.apps.anli461.qe.gcp.devcluster.openshift.com","x-forwarded-port":"443","x-forwarded-proto":"https","x-forwarded-user":"testuser-0","securitytenant":"__user__"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1","referer":"https://kibana-openshift-logging.apps.anli461.qe.gcp.devcluster.openshift.com/app/kibana"},"res":{"statusCode":403,"responseTime":27,"contentLength":9},"message":"POST /api/saved_objects/index-pattern?overwrite=false 403 27ms - 9.0B"}



Migrate from 4.5.36 to 4.5.34 is not a real custom case. I will try to get a 100% reproduce way.

Comment 44 Anping Li 2021-04-29 23:27:30 UTC
Verified on CSV elasticsearch-operator.4.6.0-202104282223.p0.

Comment 46 Matthew Robson 2021-05-03 17:05:22 UTC
*** Bug 1902112 has been marked as a duplicate of this bug. ***

Comment 50 errata-xmlrpc 2021-05-12 12:16:04 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6.28 extras update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:1489

Comment 54 Periklis Tsirakidis 2021-10-21 13:39:16 UTC
*** Bug 2005589 has been marked as a duplicate of this bug. ***

Comment 57 Red Hat Bugzilla 2023-09-18 00:25:00 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days