Previously, the logging deployer did not configure the local timezone for fluentd. The deployer has been updated to ensure new logging deployments properly configure fluentd for the local timezone.
https://access.redhat.com/solutions/2377771 details how to re-configure existing deployments
Description of problem:
In OSE 3.2.0 + RHEL 7.2, the EFK templates do not set the timezone of fluentd to the timezone of the host, leading the year detection filter to fail and filling the logs with:
"no implicit conversion of nil into String "
Full logs:
2016-06-14 06:48:52 -0400 [info]: reading config file path="/etc/fluent/fluent.conf"
2016-06-14 06:49:00 -0400 [warn]: failed to expand `%Q[#{ Time.at(time) > Time.now ? (temp_time = Time.parse(Time.at(time).to_s.gsub(Time.at(time).year.to_s, (tag_parts[3].nil? ? Time.at(time).year.to_s : tag_parts[3][9,4]) )).to_datetime.to_s; Time.parse(temp_time) > Time.now ? Time.parse(temp_time.gsub(Time.parse(temp_time).year.to_s, (Time.parse(temp_time).year - 1).to_s )).to_datetime.to_s : Time.parse(temp_time).to_datetime.to_s ) : Time.at(time).to_datetime.to_s }]` error_class=TypeError error="no implicit conversion of nil into String"
The solution is to update the deployment config to use the same timezone in the fluentd container as in the node:
# oc edit dc logging-fluentd
## add under volumeMounts:
- name: localtime
mountPath: /etc/localtime
readOnly: true
## add under volumes:
- name: localtime
hostPath:
path: /etc/localtime
Upstream fix: ff4526ad354a5d58ef6b12171e23e0b2c41b2c35
KCS: https://access.redhat.com/solutions/2377771
Slight correction: part of the fix is to the fluentd image, so that it won't spew out errors when it encounters dates in the future. So existing customers will get some benefit by updating the fluentd image. I'm not sure how that works - they'll have to redeploy fluentd with the latest image, something like oc deploy dc logging-fluentd --latest?
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHBA-2016:1608
Description of problem: In OSE 3.2.0 + RHEL 7.2, the EFK templates do not set the timezone of fluentd to the timezone of the host, leading the year detection filter to fail and filling the logs with: "no implicit conversion of nil into String " Full logs: 2016-06-14 06:48:52 -0400 [info]: reading config file path="/etc/fluent/fluent.conf" 2016-06-14 06:49:00 -0400 [warn]: failed to expand `%Q[#{ Time.at(time) > Time.now ? (temp_time = Time.parse(Time.at(time).to_s.gsub(Time.at(time).year.to_s, (tag_parts[3].nil? ? Time.at(time).year.to_s : tag_parts[3][9,4]) )).to_datetime.to_s; Time.parse(temp_time) > Time.now ? Time.parse(temp_time.gsub(Time.parse(temp_time).year.to_s, (Time.parse(temp_time).year - 1).to_s )).to_datetime.to_s : Time.parse(temp_time).to_datetime.to_s ) : Time.at(time).to_datetime.to_s }]` error_class=TypeError error="no implicit conversion of nil into String" The solution is to update the deployment config to use the same timezone in the fluentd container as in the node: # oc edit dc logging-fluentd ## add under volumeMounts: - name: localtime mountPath: /etc/localtime readOnly: true ## add under volumes: - name: localtime hostPath: path: /etc/localtime Upstream fix: ff4526ad354a5d58ef6b12171e23e0b2c41b2c35 KCS: https://access.redhat.com/solutions/2377771