Bug 1291793 - OSP Optools (Sensu) Missing Require Package On Install
OSP Optools (Sensu) Missing Require Package On Install
Product: Red Hat OpenStack
Classification: Red Hat
Component: rubygem-multi_json (Show other bugs)
7.0 (Kilo)
x86_64 Linux
low Severity low
: ga
: 9.0 (Mitaka)
Assigned To: Mike Burns
Omri Hochman
: Reopened
Depends On:
  Show dependency treegraph
Reported: 2015-12-15 10:34 EST by Benjamin Schmaus
Modified: 2016-08-11 10:10 EDT (History)
9 users (show)

See Also:
Fixed In Version: rubygem-multi_json-1.11.2-2.el7ost
Doc Type: Bug Fix
Doc Text:
This update resolves an issue whereby Sensu would fail with the error 'Encoding::UndefinedConversionError'. This would occur because Sensu can only output ASCII characters, and when systemctl is used as a check, special UTF-8 characters are used in output.
Story Points: ---
Clone Of:
Last Closed: 2016-08-11 10:10:21 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Benjamin Schmaus 2015-12-15 10:34:10 EST
Description of problem:

Customer installed Sensu from openstack-7.0-optools-rpms repos and added custom monitors but got parsing errors.

Was determined the following package was not installed when Sensu was installed:


The package above was installed as a dependency if fluentd was installed however we might want to make that package a dependency on Sensu as well

Version-Release number of selected component (if applicable): 7.0

How reproducible:

Steps to Reproduce:
1. Install Sensu
2. Add Custom Sensu Monitors
3. Restart Agent

Actual results:
Parsing error in /var/log/sensu/sensu.log:

{"timestamp":"2015-12-14T15:50:04.454605-0500","level":"warn","message":"reconnecting to transport"}
/usr/share/gems/gems/multi_json-1.10.1/lib/multi_json/adapters/json_common.rb:21:in `encode': "\xE2" from ASCII-8BIT to UTF-8 (Encoding::UndefinedConversionError)
from /usr/share/gems/gems/multi_json-1.10.1/lib/multi_json/adapters/json_common.rb:21:in `to_json'
from /usr/share/gems/gems/multi_json-1.10.1/lib/multi_json/adapters/json_common.rb:21:in `dump'
from /usr/share/gems/gems/multi_json-1.10.1/lib/multi_json/adapter.rb:24:in `dump'
from /usr/share/gems/gems/multi_json-1.10.1/lib/multi_json.rb:136:in `dump'
from /usr/share/gems/gems/sensu-logger-1.0.0/lib/sensu/logger/stream.rb:127:in `create_log_event'
from /usr/share/gems/gems/sensu-logger-1.0.0/lib/sensu/logger/stream.rb:51:in `add'
from /usr/share/gems/gems/sensu-logger-1.0.0/lib/sensu/logger/stream.rb:31:in `block (2 levels) in create_level_methods'
from /usr/share/gems/gems/sensu-0.16.0/lib/sensu/client.rb:56:in `publish_result'
from /usr/share/gems/gems/sensu-0.16.0/lib/sensu/client.rb:105:in `block in execute_check_command'
from /usr/share/gems/gems/em-worker-0.0.2/lib/em/worker.rb:21:in `call'
from /usr/share/gems/gems/em-worker-0.0.2/lib/em/worker.rb:21:in `block (2 levels) in do_task'
from /usr/share/gems/gems/eventmachine-1.0.6/lib/eventmachine.rb:960:in `call'
from /usr/share/gems/gems/eventmachine-1.0.6/lib/eventmachine.rb:960:in `run_deferred_callbacks'
from /usr/share/gems/gems/eventmachine-1.0.6/lib/eventmachine.rb:187:in `run_machine'
from /usr/share/gems/gems/eventmachine-1.0.6/lib/eventmachine.rb:187:in `run'
from /usr/share/gems/gems/sensu-0.16.0/lib/sensu/client.rb:12:in `run'
from /usr/share/gems/gems/sensu-0.16.0/bin/sensu-client:10:in `<top (required)>'
from /usr/bin/sensu-client:23:in `load'
from /usr/bin/sensu-client:23:in `<main>'

Expected results:
No errors

Additional info:

Customer had not installed fluentd on the host.
Comment 3 Martin Magr 2016-03-08 11:43:31 EST
This requirement IMHO should be added to rubygem-multi_json [1], which is interface to set of the JSON parsing library, which is used by Sensu.

[1] https://rubygems.org/gems/multi_json
Comment 5 Martin Magr 2016-04-07 08:22:24 EDT
Rubygem yajl-ruby is not required dependency of any rubygem on which sensu is dependent on (not even of multi_json) in the end. yajl-ruby is one of the JSON parsing library which can be used by multi_json. As installation of the rubygem can fix the customer's issue, it is not required, because multi_json is bundled with okJson library as default JSON parsing library, which should be enough to parse json files with checks. So the original cause is IMHO hidden somewhere else.

I've been investigating a little bit and trying to reproduce the issue. Considering the error is Encoding::UndefinedConversionError and command with which customer created check JSON files can generate checks for below services. Most important is the "●", which basically mean that check-●.json for checking service ● will be created. Can customer verify properly all the generated json files? Special characters should not be used within the files.  

[root@controller ~]# for i in $(systemctl | egrep 'openstack|neutron' | grep -v losetup | awk '{print $1}'); do echo $i; done
[root@controller ~]#
Comment 6 Benjamin Schmaus 2016-04-07 08:31:28 EDT

"Can customer verify properly all the generated json files? Special characters should not be used within the files."

Customer had another case where they also ran into unicode errors related to OpenStack.  Turns out they were copy and pasting from a document.   I wonder given your questioning if that is the case here as well, that they copied out of a document and pasted into json file and the result was some non visible characters embedded in json.
Comment 7 Martin Magr 2016-04-07 09:27:06 EDT
Ok, I've reproduced the issue with the any of the generated JSON file. So the exception is not raised because Sensu cannot read JSON configuration files, but because systemctl output contains utf-8 special characters which are then failed to be parsed. Customer should update their checks to suppress the systemctl output.
Comment 8 Martin Magr 2016-04-07 09:35:36 EDT
@Benjamin: When I was trying to run check-●.json and Sensu failed to start with message that check file is containing special character. So copy/pasting should be OK, because user will be warned when the JSON file is unconvertable. The only problem I see now is that the output to stdout/stderr should not contain any special characters then Sensu is working without problem for me.
Comment 9 Martin Magr 2016-04-12 05:04:08 EDT
After discussion within the team, we decided to actually add rubygem-yajl-ruby as dependency of rubygem-multi_json
Comment 14 errata-xmlrpc 2016-08-11 10:10:21 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.