Bug 1366137

Summary: Fluentd main process died unexpectedly because it could not find 'mime-types' (< 4.0, >= 1.16) among 44 total gem(s) (Gem::LoadError)
Product: OpenShift Container Platform Reporter: Xia Zhao <xiazhao>
Component: LoggingAssignee: Rich Megginson <rmeggins>
Status: CLOSED ERRATA QA Contact: chunchen <chunchen>
Severity: high Docs Contact:
Priority: high    
Version: 3.3.0CC: aos-bugs, ewolinet, jcantril, tdawson, wsun
Target Milestone: ---Keywords: Regression, TestBlocker
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
No doc update needed - this was found by QE before being shipped to customers.
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-09-27 09:43:33 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
deployer_pod_log
none
fluentd_pod_log
none
fluentd_log_image_5a5484ab509e none

Comment 1 Xia Zhao 2016-08-11 07:46:54 UTC
Created attachment 1189931 [details]
deployer_pod_log

Comment 2 Xia Zhao 2016-08-11 07:47:34 UTC
Created attachment 1189932 [details]
fluentd_pod_log

Comment 3 Rich Megginson 2016-08-12 20:54:08 UTC
Fixed with this build:

  https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=508857
repositories:
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-fluentd:rhaos-3.3-rhel-7-docker-candidate-20160812203857
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-fluentd:3.3.0-13
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-fluentd:3.3.0
  brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/openshift3/logging-fluentd:latest

Comment 5 Xia Zhao 2016-08-15 05:27:34 UTC
Created attachment 1190790 [details]
fluentd_log_image_5a5484ab509e

Comment 6 Xia Zhao 2016-08-16 05:55:21 UTC
Reproduced with fluentd image 3.3.0-14 on brew registry:

The error message is changed:

$ oc logs -f logging-fluentd-u4xnr
2016-08-16 05:53:57 +0000 [info]: reading config file path="/etc/fluent/fluent.conf"
/usr/share/rubygems/rubygems/dependency.rb:296:in `to_specs': Could not find 'unf' (< 1.0.0, >= 0.0.5) among 42 total gem(s) (Gem::LoadError)
	from /usr/share/rubygems/rubygems/specification.rb:1209:in `block in activate_dependencies'
	from /usr/share/rubygems/rubygems/specification.rb:1198:in `each'
	from /usr/share/rubygems/rubygems/specification.rb:1198:in `activate_dependencies'
	from /usr/share/rubygems/rubygems/specification.rb:1180:in `activate'
	from /usr/share/rubygems/rubygems/specification.rb:1212:in `block in activate_dependencies'
	from /usr/share/rubygems/rubygems/specification.rb:1198:in `each'
	from /usr/share/rubygems/rubygems/specification.rb:1198:in `activate_dependencies'
	from /usr/share/rubygems/rubygems/specification.rb:1180:in `activate'
	from /usr/share/rubygems/rubygems/specification.rb:1212:in `block in activate_dependencies'
	from /usr/share/rubygems/rubygems/specification.rb:1198:in `each'
	from /usr/share/rubygems/rubygems/specification.rb:1198:in `activate_dependencies'
	from /usr/share/rubygems/rubygems/specification.rb:1180:in `activate'
	from /usr/share/rubygems/rubygems/specification.rb:1212:in `block in activate_dependencies'
	from /usr/share/rubygems/rubygems/specification.rb:1198:in `each'
	from /usr/share/rubygems/rubygems/specification.rb:1198:in `activate_dependencies'
	from /usr/share/rubygems/rubygems/specification.rb:1180:in `activate'
	from /usr/share/rubygems/rubygems.rb:186:in `rescue in try_activate'
	from /usr/share/rubygems/rubygems.rb:183:in `try_activate'
	from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:132:in `rescue in require'
	from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:144:in `require'
	from /usr/share/gems/gems/fluent-plugin-kubernetes_metadata_filter-0.24.0/lib/fluent/plugin/filter_kubernetes_metadata.rb:105:in `configure'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/agent.rb:141:in `add_filter'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/agent.rb:58:in `block in configure'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/agent.rb:54:in `each'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/agent.rb:54:in `configure'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/root_agent.rb:79:in `block in configure'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/root_agent.rb:79:in `each'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/root_agent.rb:79:in `configure'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/engine.rb:117:in `configure'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/engine.rb:91:in `run_configure'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/supervisor.rb:515:in `run_configure'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/supervisor.rb:146:in `block in start'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/supervisor.rb:352:in `call'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/supervisor.rb:352:in `main_process'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/supervisor.rb:325:in `block in supervise'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/supervisor.rb:324:in `fork'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/supervisor.rb:324:in `supervise'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/supervisor.rb:142:in `start'
	from /usr/share/gems/gems/fluentd-0.12.20/lib/fluent/command/fluentd.rb:171:in `<top (required)>'
	from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /usr/share/gems/gems/fluentd-0.12.20/bin/fluentd:6:in `<top (required)>'
	from /usr/bin/fluentd:23:in `load'
	from /usr/bin/fluentd:23:in `<main>'
2016-08-16 05:53:58 +0000 [warn]: process died within 1 second. exit.

Comment 9 Rich Megginson 2016-08-18 19:43:07 UTC
It turns out that there are a lot more missing dependencies.  Please don't test or update this bug until I get them all sorted out.  I will move this bug back to ON_QA when I have, once and for all, verified that all of the dependencies are present.  Sorry for the noise and churn.  I realize this is a test blocker and I'm working non-stop on this until it is resolved.

Comment 13 errata-xmlrpc 2016-09-27 09:43:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1933