Bug 1365783

Summary: Fluentd pod not able to startup because of error="Unknown output plugin 'rewrite_tag_filter' when using images from registry.ops
Product: OpenShift Container Platform Reporter: Xia Zhao <xiazhao>
Component: LoggingAssignee: Rich Megginson <rmeggins>
Status: CLOSED ERRATA QA Contact: chunchen <chunchen>
Severity: high Docs Contact:
Priority: high    
Version: 3.3.0CC: aos-bugs, rmeggins, tdawson, wsun, xiazhao, xtian
Target Milestone: ---Keywords: Regression
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
No doc update needed - this was found by QE before being shipped to customers.
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-09-27 09:43:24 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Comment 5 Xia Zhao 2016-08-11 06:13:29 UTC
Set to verified according to comment #4

Comment 9 Rich Megginson 2016-08-12 20:58:17 UTC
Troy, how can we reduce this churn?  The problem is that I fix a bug and build the container with the unsigned repo, then someone else builds the container with the signed repo that doesn't have the package in it yet, or doesn't have the right version yet, and the bug gets reopened (or another bug gets opened).

The version of the fluentd container that has all of the correct packages is this:

  https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=508857
repositories:
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-fluentd:rhaos-3.3-rhel-7-docker-candidate-20160812203857
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-fluentd:3.3.0-13
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-fluentd:3.3.0
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-fluentd:latest

This was built using 
rhpkg container-build --repo=http://file.rdu.redhat.com/tdawson/repo/aos-unsigned-building.repo

If this is not the correct way to build containers for QE to test, please let me know what is the correct procedure.

Comment 10 Troy Dawson 2016-08-15 20:11:21 UTC
For this particular problem, this happened to a problem pushing to registry.ops last friday.  From our end, it looked like our images were pushed, but the cluster was having issues and when the cluster was fixes, many of the images we thought we had pushed, were not there.  This should be fixed now.

Comment 11 Xia Zhao 2016-08-17 05:56:10 UTC
@tdawson Yes, checked with the latest image on ops registry, the original issue in this bz no longer exist, however , encounter the reproduced of https://bugzilla.redhat.com/show_bug.cgi?id=1366137 :


/usr/share/rubygems/rubygems/dependency.rb:296:in `to_specs': Could not find 'domain_name' (~> 0.5) among 41 total gem(s) (Gem::LoadError)
	from /usr/share/rubygems/rubygems/specification.rb:1209:in `block in activate_dependencies'
	from /usr/share/rubygems/rubygems/specification.rb:1198:in `each'
	from /usr/share/rubygems/rubygems/specification.rb:1198:in `activate_dependencies'
	from /usr/share/rubygems/rubygems/specification.rb:1180:in `activate'
	from /usr/share/rubygems/rubygems/specification.rb:1212:in `block in activate_dependencies'
	from /usr/share/rubygems/rubygems/specification.rb:1198:in `each'
	from /usr/share/rubygems/rubygems/specification.rb:1198:in `activate_dependencies'
	from /usr/share/rubygems/rubygems/specification.rb:1180:in `activate'
	from /usr/share/rubygems/rubygems/specification.rb:1212:in `block in activate_dependencies'
	from /usr/share/rubygems/rubygems/specification.rb:1198:in `each'
	from /usr/share/rubygems/rubygems/specification.rb:1198:in `activate_dependencies'
	from /usr/share/rubygems/rubygems/specification.rb:1180:in `activate'
	from /usr/share/rubygems/rubygems.rb:186:in `rescue in try_activate'
	from /usr/share/rubygems/rubygems.rb:183:in `try_activate'
	from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:132:in `rescue in require'
	from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:144:in `require'
	from /usr/share/gems/gems/fluent-plugin-kubernetes_metadata_filter-0.24.0/lib/fluent/plugin/filter_kubernetes_metadata.rb:105:in `configure'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/agent.rb:141:in `add_filter'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/agent.rb:58:in `block in configure'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/agent.rb:54:in `each'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/agent.rb:54:in `configure'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/root_agent.rb:79:in `block in configure'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/root_agent.rb:79:in `each'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/root_agent.rb:79:in `configure'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/engine.rb:117:in `configure'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/engine.rb:91:in `run_configure'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/supervisor.rb:515:in `run_configure'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/supervisor.rb:146:in `block in start'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/supervisor.rb:352:in `call'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/supervisor.rb:352:in `main_process'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/supervisor.rb:325:in `block in supervise'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/supervisor.rb:324:in `fork'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/supervisor.rb:324:in `supervise'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/supervisor.rb:142:in `start'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/command/fluentd.rb:171:in `<top (required)>'
	from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /usr/share/gems/gems/fluentd-0.12.20/bin/fluentd:6:in `<top (required)>'
	from /usr/bin/fluentd:23:in `load'
	from /usr/bin/fluentd:23:in `<main>'
2016-08-17 01:51:47 -0400 [warn]: process died within 1 second. exit.

Comment 12 Xia Zhao 2016-08-17 06:00:34 UTC
@tdawson Based on comment #11, please feel free to assign this bz back to ON_QA for closure, the fluentd problem can be tracked in another testblocker bz: 1366137.

Comment 13 Xia Zhao 2016-08-18 01:01:19 UTC
Set to verified according to comment #11

Comment 15 errata-xmlrpc 2016-09-27 09:43:24 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1933