Bug 1523047

Summary: [3.9] exception output while using view archive link in pod log for default project
Product: OpenShift Container Platform Reporter: Rajnikant <rkant>
Component: LoggingAssignee: Jeff Cantrill <jcantril>
Status: CLOSED ERRATA QA Contact: Junqi Zhao <juzhao>
Severity: low Docs Contact:
Priority: low    
Version: 3.6.1CC: aos-bugs, dmoessne, jcantril, jmalde, juzhao, pprakash, rkant, rmeggins
Target Milestone: ---Flags: jcantril: needinfo-
Target Release: 3.9.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: The link generation code assumes all project logs are written to indices that have a common naming pattern. Consequence: User's are linked to non-existent indices Fix: Project logs that will be archived to different indices are annotated with the required information to properly build the link. Result: Users are routed using a link that will query the data store correctly and return data.
Story Points: ---
Clone Of:
: 1547348 1547688 (view as bug list) Environment:
Last Closed: 2018-08-09 22:13:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1525357    
Bug Blocks: 1547348, 1547688    
Attachments:
Description Flags
click "view archive" link in default project -- non-ops cluster
none
click "view archive" link in default project -- ops enabled cluster
none
Issue is reproduced, waring shows again none

Comment 1 Junqi Zhao 2017-12-15 02:34:24 UTC
OCP env set up work is blocked by 
https://bugzilla.redhat.com/show_bug.cgi?id=1525357

Will verify it after BZ # 1525357 is fixed

Comment 2 Jeff Cantrill 2017-12-15 14:13:03 UTC
Moving this back to 'new'.  The issue here is there are certain projects where the logs go to the 'operations' index which is the case here.  The webconsole does not take that into account.

Comment 4 openshift-github-bot 2018-01-19 05:52:00 UTC
Commits pushed to master at https://github.com/openshift/openshift-ansible

https://github.com/openshift/openshift-ansible/commit/d0fed2b6d24ddbedbc33acffd1c9decf65a54e66
bug 1523047. Annotate ops projects with an .operation prefix

https://github.com/openshift/openshift-ansible/commit/32b1287b26c7a3e047d51fd63b5ab3a08fa4c0f1
Merge pull request #6765 from jcantrill/bz1523047_annotate_logging_prefix

Automatic merge from submit-queue.

bug 1523047. Annotate ops projects with an .operation prefix

Comment 5 openshift-github-bot 2018-01-19 14:55:10 UTC
Commits pushed to master at https://github.com/openshift/origin-web-console

https://github.com/openshift/origin-web-console/commit/4115a78cabf3c5cc05ef645e4e8aa29256158447
bug 1523047. fix log archive link for operation namespaces

https://github.com/openshift/origin-web-console/commit/5dcf858df5b48e96ec84d30117df719ba1d77f50
Merge pull request #2678 from jcantrill/1523047_kibana_link_for_ops

Automatic merge from submit-queue.

bug 1523047. fix log archive link for operation namespaces

Comment 7 Junqi Zhao 2018-01-23 06:02:48 UTC
Tested with ose-web-console/images/v3.9.0-0.22.0.0, covered non-ops and ops enabled cluster, when click "view archive" link in pod log, it can navigate to kibana UI without error throw out in kibana UI.

See the attached pictures.

Comment 8 Junqi Zhao 2018-01-23 06:05:07 UTC
Created attachment 1384682 [details]
click "view archive" link in default project -- non-ops cluster

Comment 9 Junqi Zhao 2018-01-23 06:05:35 UTC
Created attachment 1384683 [details]
click "view archive" link in default project -- ops enabled cluster

Comment 10 Junqi Zhao 2018-01-24 05:57:56 UTC
Verified with fluentd 3.7 images, other component used 3.9 images since defect
https://bugzilla.redhat.com/show_bug.cgi?id=1531157

Comment 14 Jeff Cantrill 2018-02-21 14:15:34 UTC
Backport to 3.7 targeted to https://bugzilla.redhat.com/show_bug.cgi?id=1547348

Comment 16 Junqi Zhao 2018-03-08 02:09:23 UTC
Issue is reproduced, see the attached picture, click "View Archive" link from default project, the warning shows again in Kibana UI, the index is .all, should use .operations.* index in Kibana UI


Warning
Discover: "project.default.47990a88-21d8-11e8-a453-fa163e5fe7e9.*" is not a configured pattern. Using the default index pattern: ".all"

All logging component version is v3.9.3-1

Comment 17 Junqi Zhao 2018-03-08 02:10:11 UTC
Created attachment 1405662 [details]
Issue is reproduced, waring shows again

Comment 18 Junqi Zhao 2018-03-08 02:40:07 UTC
(In reply to Junqi Zhao from comment #16)
> Issue is reproduced, see the attached picture, click "View Archive" link
> from default project, the warning shows again in Kibana UI, the index is
> .all, should use .operations.* index in Kibana UI
> 
> 
> Warning
> Discover: "project.default.47990a88-21d8-11e8-a453-fa163e5fe7e9.*" is not a
> configured pattern. Using the default index pattern: ".all"
> 
> All logging component version is v3.9.3-1

env:
# openshift version
openshift v3.9.3
kubernetes v1.9.1+a0ce1bc657
etcd 3.2.16

deployed logging without ops cluster enabled

Comment 19 Jeff Cantrill 2018-03-08 14:57:37 UTC
Please provide the project annotations: `oc get project default -o yaml'.

The annotations provide hints to the webconsole to properly build the url.  If the new image was deployed without running ansible, you may not have executed the code to annotate the project.  Additionally, please provide the version of the webconsole image so we can confirm it also has the code that consumes the annotation.

Comment 20 daniel 2018-03-10 17:16:50 UTC
backport to 3.6 Bug 1547688 - [3.6] exception output while using view archive link in pod log for default project

Comment 22 Jeff Cantrill 2018-03-13 13:56:36 UTC
Can you advise if this logging deployment additionally uses an operations instance of the logging stack too?  It looks like we still may have an error where the annotation is not applied if the ops cluster is not deployed [1]

[1] https://github.com/openshift/openshift-ansible/blob/release-3.9/roles/openshift_logging/tasks/annotate_ops_projects.yaml#L15

Comment 23 Jeff Cantrill 2018-03-14 13:30:11 UTC
The merge of https://github.com/openshift/openshift-ansible/pull/7522  allows properly annotating projects for the prefix regardless if the 'ops' EFK stack is deployed.  Needs a new ansible release to resolve once merged.

Manual work around is to annotation the default project with:

openshift.io/logging.data.prefix: ".operations"

Comment 26 Junqi Zhao 2018-07-30 06:36:00 UTC
logging component images version: v3.9.38
Tested with enabled ops cluster, it will navigate to kibana-ops UI without error
Tested with disabled ops cluster, it will navigate to kibana UI without error
the index pattern is .operations.* in both scenarios

Comment 28 errata-xmlrpc 2018-08-09 22:13:46 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:2335