Bug 1511719 - operations logs lost if no ops cluster and no @OUTPUT label
Summary: operations logs lost if no ops cluster and no @OUTPUT label
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Logging
Version: 3.7.0
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: 3.7.0
Assignee: Rich Megginson
QA Contact: Anping Li
URL:
Whiteboard:
Depends On:
Blocks: 1512028
TreeView+ depends on / blocked
 
Reported: 2017-11-10 00:02 UTC by Rich Megginson
Modified: 2017-12-18 13:23 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: When upgrading to a new version of logging, if the deployment does not use a separate OPS logging cluster, and there is no @OUTPUT label in the fluent.conf, fluentd does not know how to route the operations logs. Consequence: The operations logs are not logged. User will see no new operations (.operations.*) logs after the upgrade. Fix: Change the fluentd configuration to correctly handle the case both with and without the @OUTPUT label, and for the OPS and non-OPS cases. Result: Operations logs flow uninterrupted after upgrade.
Clone Of:
: 1512028 (view as bug list)
Environment:
Last Closed: 2017-12-18 13:23:56 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift origin-aggregated-logging pull 774 0 None None None 2017-11-10 00:03:56 UTC
Red Hat Product Errata RHBA-2017:3464 0 normal SHIPPED_LIVE Red Hat OpenShift Container Platform 3.7 bug fix and enhancement update 2017-12-18 18:22:05 UTC

Description Rich Megginson 2017-11-10 00:02:51 UTC 
Description of problem:
Deploy logging with no ops cluster and based on 3.6 or an earlier 3.7 pre-release which does not have the @OUTPUT label in fluent.conf

update to a recent fluentd image

no operations logs are stored

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
Comment 1 Rich Megginson 2017-11-10 17:20:45 UTC 
https://github.com/openshift/origin-aggregated-logging/commit/61d1196eeab4ac02b360b20f9338d3ab2b5d5eef
Comment 2 Rich Megginson 2017-11-11 00:19:18 UTC 
koji_builds:
  https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=623852
repositories:
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-fluentd:rhaos-3.7-rhel-7-docker-candidate-29398-20171110235311
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-fluentd:latest
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-fluentd:v3.7.6
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-fluentd:v3.7.6-2
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-fluentd:v3.7
Comment 3 Anping Li 2017-11-14 10:00:04 UTC 
@Rich,
With logging-fluentd/images/v3.7.7-1. The .operations logs can be found both mux is enabled and disabled when OUPPUT label is removed. I think the test result is pass.

The bug is move from ON_QA to Modified, Is there other fix?
Comment 4 Rich Megginson 2017-11-14 14:34:54 UTC 
(In reply to Anping Li from comment #3)
> @Rich,
> With logging-fluentd/images/v3.7.7-1. The .operations logs can be found both
> mux is enabled and disabled when OUPPUT label is removed. I think the test
> result is pass.
> 
> The bug is move from ON_QA to Modified, Is there other fix?

Did you deploy without OPS cluster?  The bug can only be reproduced when you deploy logging without OPS cluster. If you did not deploy OPS cluster, and you could not reproduce the issue, then the bug is verified.
Comment 5 Anping Li 2017-11-15 01:54:46 UTC 
Yes, I had Deployed without OPS cluster.  so move to verified.
Comment 8 errata-xmlrpc 2017-12-18 13:23:56 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:3464
Note You need to log in before you can comment on or make changes to this bug.