Bug 1472060 - [dedicated] dedicated admin should have a way to access to kibana
[dedicated] dedicated admin should have a way to access to kibana
Status: NEW
Product: OpenShift Online
Classification: Red Hat
Component: RFE (Show other bugs)
3.x
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Samuel Padgett
yapei
: OnlineDedicated
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-07-17 22:01 EDT by Kenjiro Nakayama
Modified: 2017-08-17 02:25 EDT (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Kenjiro Nakayama 2017-07-17 22:01:47 EDT
Description of problem:
---
- Currently, dedicated admin cannot access to view logs via kibana directly. They have to be routed through one of the Pod's "view archive" button.

  OK: webconsole->Applications->pod->Logs->View Archive->Kibana
  NG: kibana URL directory (e.g https://logs.ded-stage-aws.openshift.com/) due to getting an error

- This bothers users (especially admin who wants to view whole apps logs) with redundant operation. We would like webconsole's "Project" top page to have an button to access kibana.

Version-Release number of selected component (if applicable):
---
- OpenShift Dedicated

How reproducible:
---
- 100%

Steps to Reproduce:
---
1. Access to kibana URL without routing through pod's "view archive" https://logs.ded-stage-aws.openshift.com/

Actual results:
---
- Failed to access due to an error.

Expected results:
---
- each projects has an archive button for kibana console.
- Please refer to [1] as well.

[1] http://post-office.corp.redhat.com/archives/rhose-prio-list/2017-July/msg00065.html
"In the mean time, if a user picks any project to view first, then land in kibana via view archive button, they will have a pull down list of all the projects to view and should be able to view them."
Comment 1 Jessica Forrester 2017-07-18 08:34:51 EDT
Are you asking for a link to view logs for a particular project, or a completely top level link that shows all of the projects?
Comment 2 Kenjiro Nakayama 2017-07-18 09:05:30 EDT
> Are you asking for a link to view logs for a particular project, or a completely top level link that shows all of the projects?

I think later (completely top level link) is better.

Let me explain the reason. Currently if we go to kibana console from "app1" pod in "p1" project, kibana will search:

  'kubernetes.pod_name:"app1" AND kubernetes.namespace_name:"p1"'

and kibana does NOT allow us to view any apps's logs in other project (even same user's). In other words, if same user has a "p2" project, we have to route through a pod in "p2" project. So, I think completely top level link could provide users with more flexible search in kibana.
Comment 3 Jeff Cantrill 2017-07-18 09:41:15 EDT
The link provided from the web UI is simply a convenience to retrieve pod logs.  You can manually select the namespace for a different project to which you have access; there is nothing restricting you from doing that.  You can set up mappings to your indexes through the settings tab of Kibana which will allow you to query any namespace.  The plugins that are included in the logging stack will set up mappings for all your projects by default when you first log in.  The only known issue is trying to query logs from pods in multiple namespaces.  Our design needs to change to allow that to be possible.
Comment 4 Kenjiro Nakayama 2017-07-19 04:43:06 EDT
> The link provided from the web UI is simply a convenience to retrieve pod logs.  You can manually select the namespace for a different project to which you have access; there is nothing restricting you from doing that.

If we accessed kibana (logs.ded-stage-aws.openshift.com) directory, it does NOT allow me to search *any* logs due to "Discover: [security_exception] no permissions for indices:data/read/msearch". Then, I heard that it is because we have to access to the kibana via OpenShift webconsole as OpenShift set the proper cookies[1]. That's why I am asking you to add link to the OpenShift webconsole.

> You can set up mappings to your indexes through the settings tab of Kibana which will allow you to query any namespace.  The plugins that are included in the logging stack will set up mappings for all your projects by default when you first log in.

So, this will fix the "no permissions for indices:data/read/msearch" error, which I mentioned above? I have tried it, but didn't work. Could you please tell us one of the example index settings which should work?

[1] http://post-office.corp.redhat.com/archives/rhose-prio-list/2017-July/msg00063.html
"Unless OpenShift has set the proper cookies in the browser used to load the Kibana web pages, you won't have proper access to view logs.  The only reliable way to view logs is to go to kibana from the console page."
Comment 5 Kenjiro Nakayama 2017-08-02 03:50:00 EDT
@Jessica if the logging side issue is that user cannot view multiple namespaces, could you please add a link to view logs for a particular project?

As I mentioned in the first comment, current operation to view kibana console "webconsole->Applications->pod->Logs->View Archive->Kibana" gets admin users frustrated.
Comment 6 Samuel Padgett 2017-08-08 10:33:17 EDT
Yes, adding a link for logs in a particular project makes sense to me.

Jeff, is it possible for the web console to provide this link? What would the URL need to be?

This is the template we're using currently for pod logs.

https://github.com/openshift/origin-web-console/blob/master/app/scripts/services/logLinks.js#L57-L82
Comment 7 Jeff Cantrill 2017-08-08 10:47:14 EDT
(In reply to Kenjiro Nakayama from comment #4)

> If we accessed kibana (logs.ded-stage-aws.openshift.com) directly, it does
> NOT allow me to search *any* logs due to "Discover: [security_exception] no
> permissions for indices:data/read/msearch". Then, I heard that it is because
> we have to access to the kibana via OpenShift webconsole as OpenShift set
> the proper cookies[1]. That's why I am asking you to add link to the
> OpenShift webconsole.
> 
This exception is misleading and is not really a security exception.  You will see this issue if no logs are available for the project.  This means there are no indices in Elasticsearch that have the pattern 'project.<PROJECTNAME>.<DATEPATTERN>'.  The security plugin we use is unable to find metadata for the index in question and instead of returning a 404 improperly throws a security exception.  This fix was added with https://bugzilla.redhat.com/show_bug.cgi?id=1456413 and is available in v3.6.126-2 or greater. We have additionally backported to previously releases depending upon what version online is using.  



> > You can set up mappings to your indexes through the settings tab of Kibana which will allow you to query any namespace.  The plugins that are included in the logging stack will set up mappings for all your projects by default when you first log in.
> 
> So, this will fix the "no permissions for indices:data/read/msearch" error,
> which I mentioned above? I have tried it, but didn't work. Could you please
> tell us one of the example index settings which should work?

This referance change additionally brings in logic that only creates mappings for your projects which also have at least 1 index in Elasticsearch.  If you are unable to access your logs, it is most likely that there are no logs to access.



(In reply to Samuel Padgett from comment #6)
> Yes, adding a link for logs in a particular project makes sense to me.
> 
> Jeff, is it possible for the web console to provide this link? What would
> the URL need to be?
> 
We need to make this easier since the console is dependent on kibana specific fields.  The URL to my project is like [1].  Looking at your code I think we would need to only remove the 'container' info since we would not have a specific container.

[1] https://kibana.192.168.122.61.nip.io/app/kibana#/discover?_g=(refreshInterval:(display:Off,pause:!f,value:0),time:(from:now-15m,mode:quick,to:now))&_a=(columns:!(_source),index:'project.logging.4e2bc514-7884-11e7-b5af-52540050d5ea.*',interval:auto,query:(query_string:(analyze_wildcard:!t,query:'*')),sort:!('@timestamp',desc))

Note You need to log in before you can comment on or make changes to this bug.