Bug 1291793

Summary: OSP Optools (Sensu) Missing Require Package On Install
Product: Red Hat OpenStack Reporter: Benjamin Schmaus <bschmaus>
Component: rubygem-multi_jsonAssignee: Mike Burns <mburns>
Status: CLOSED ERRATA QA Contact: Omri Hochman <ohochman>
Severity: low Docs Contact:
Priority: low    
Version: 7.0 (Kilo)CC: adahms, apevec, jjoyce, lhh, markmc, mburns, mmagr, rhos-maint, srevivo
Target Milestone: gaKeywords: Reopened
Target Release: 9.0 (Mitaka)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: rubygem-multi_json-1.11.2-2.el7ost Doc Type: Bug Fix
Doc Text:
This update resolves an issue whereby Sensu would fail with the error 'Encoding::UndefinedConversionError'. This would occur because Sensu can only output ASCII characters, and when systemctl is used as a check, special UTF-8 characters are used in output.
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-08-11 14:10:21 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Benjamin Schmaus 2015-12-15 15:34:10 UTC
Description of problem:

Customer installed Sensu from openstack-7.0-optools-rpms repos and added custom monitors but got parsing errors.

Was determined the following package was not installed when Sensu was installed:

rubygem-yajl-ruby

The package above was installed as a dependency if fluentd was installed however we might want to make that package a dependency on Sensu as well

Version-Release number of selected component (if applicable): 7.0

How reproducible:
100%

Steps to Reproduce:
1. Install Sensu
2. Add Custom Sensu Monitors
3. Restart Agent

Actual results:
Parsing error in /var/log/sensu/sensu.log:

{"timestamp":"2015-12-14T15:50:04.454605-0500","level":"warn","message":"reconnecting to transport"}
/usr/share/gems/gems/multi_json-1.10.1/lib/multi_json/adapters/json_common.rb:21:in `encode': "\xE2" from ASCII-8BIT to UTF-8 (Encoding::UndefinedConversionError)
from /usr/share/gems/gems/multi_json-1.10.1/lib/multi_json/adapters/json_common.rb:21:in `to_json'
from /usr/share/gems/gems/multi_json-1.10.1/lib/multi_json/adapters/json_common.rb:21:in `dump'
from /usr/share/gems/gems/multi_json-1.10.1/lib/multi_json/adapter.rb:24:in `dump'
from /usr/share/gems/gems/multi_json-1.10.1/lib/multi_json.rb:136:in `dump'
from /usr/share/gems/gems/sensu-logger-1.0.0/lib/sensu/logger/stream.rb:127:in `create_log_event'
from /usr/share/gems/gems/sensu-logger-1.0.0/lib/sensu/logger/stream.rb:51:in `add'
from /usr/share/gems/gems/sensu-logger-1.0.0/lib/sensu/logger/stream.rb:31:in `block (2 levels) in create_level_methods'
from /usr/share/gems/gems/sensu-0.16.0/lib/sensu/client.rb:56:in `publish_result'
from /usr/share/gems/gems/sensu-0.16.0/lib/sensu/client.rb:105:in `block in execute_check_command'
from /usr/share/gems/gems/em-worker-0.0.2/lib/em/worker.rb:21:in `call'
from /usr/share/gems/gems/em-worker-0.0.2/lib/em/worker.rb:21:in `block (2 levels) in do_task'
from /usr/share/gems/gems/eventmachine-1.0.6/lib/eventmachine.rb:960:in `call'
from /usr/share/gems/gems/eventmachine-1.0.6/lib/eventmachine.rb:960:in `run_deferred_callbacks'
from /usr/share/gems/gems/eventmachine-1.0.6/lib/eventmachine.rb:187:in `run_machine'
from /usr/share/gems/gems/eventmachine-1.0.6/lib/eventmachine.rb:187:in `run'
from /usr/share/gems/gems/sensu-0.16.0/lib/sensu/client.rb:12:in `run'
from /usr/share/gems/gems/sensu-0.16.0/bin/sensu-client:10:in `<top (required)>'
from /usr/bin/sensu-client:23:in `load'
from /usr/bin/sensu-client:23:in `<main>'

Expected results:
No errors

Additional info:

Customer had not installed fluentd on the host.

Comment 3 Martin Magr 2016-03-08 16:43:31 UTC
This requirement IMHO should be added to rubygem-multi_json [1], which is interface to set of the JSON parsing library, which is used by Sensu.

[1] https://rubygems.org/gems/multi_json

Comment 5 Martin Magr 2016-04-07 12:22:24 UTC
Rubygem yajl-ruby is not required dependency of any rubygem on which sensu is dependent on (not even of multi_json) in the end. yajl-ruby is one of the JSON parsing library which can be used by multi_json. As installation of the rubygem can fix the customer's issue, it is not required, because multi_json is bundled with okJson library as default JSON parsing library, which should be enough to parse json files with checks. So the original cause is IMHO hidden somewhere else.

I've been investigating a little bit and trying to reproduce the issue. Considering the error is Encoding::UndefinedConversionError and command with which customer created check JSON files can generate checks for below services. Most important is the "●", which basically mean that check-●.json for checking service ● will be created. Can customer verify properly all the generated json files? Special characters should not be used within the files.  

[root@controller ~]# for i in $(systemctl | egrep 'openstack|neutron' | grep -v losetup | awk '{print $1}'); do echo $i; done
neutron-dhcp-agent.service
neutron-l3-agent.service
neutron-metadata-agent.service
neutron-openvswitch-agent.service
neutron-server.service
openstack-cinder-api.service
openstack-cinder-backup.service
openstack-cinder-scheduler.service
openstack-cinder-volume.service
openstack-glance-api.service
openstack-glance-registry.service
●
openstack-nova-api.service
openstack-nova-cert.service
openstack-nova-conductor.service
openstack-nova-consoleauth.service
openstack-nova-novncproxy.service
openstack-nova-scheduler.service
openstack-swift-account-auditor.service
openstack-swift-account-reaper.service
openstack-swift-account-replicator.service
openstack-swift-account.service
openstack-swift-container-auditor.service
openstack-swift-container-replicator.service
openstack-swift-container-updater.service
openstack-swift-container.service
openstack-swift-object-auditor.service
openstack-swift-object-expirer.service
openstack-swift-object-replicator.service
openstack-swift-object-updater.service
openstack-swift-object.service
openstack-swift-proxy.service
[root@controller ~]#

Comment 6 Benjamin Schmaus 2016-04-07 12:31:28 UTC
@Martin

"Can customer verify properly all the generated json files? Special characters should not be used within the files."

Customer had another case where they also ran into unicode errors related to OpenStack.  Turns out they were copy and pasting from a document.   I wonder given your questioning if that is the case here as well, that they copied out of a document and pasted into json file and the result was some non visible characters embedded in json.

Comment 7 Martin Magr 2016-04-07 13:27:06 UTC
Ok, I've reproduced the issue with the any of the generated JSON file. So the exception is not raised because Sensu cannot read JSON configuration files, but because systemctl output contains utf-8 special characters which are then failed to be parsed. Customer should update their checks to suppress the systemctl output.

Comment 8 Martin Magr 2016-04-07 13:35:36 UTC
@Benjamin: When I was trying to run check-●.json and Sensu failed to start with message that check file is containing special character. So copy/pasting should be OK, because user will be warned when the JSON file is unconvertable. The only problem I see now is that the output to stdout/stderr should not contain any special characters then Sensu is working without problem for me.

Comment 9 Martin Magr 2016-04-12 09:04:08 UTC
After discussion within the team, we decided to actually add rubygem-yajl-ruby as dependency of rubygem-multi_json

Comment 14 errata-xmlrpc 2016-08-11 14:10:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-1600.html