Description of problem: A couple of times a day the customer gets an ABRT message regarding the nagios plugin check_cluster_vol_usage.py: abrt_version: 2.0.8 cmdline: /usr/bin/python /usr/lib64/nagios/plugins/gluster/check_cluster_vol_usage.py -w 80 -c 90 -hg Default event_log: executable: /usr/lib64/nagios/plugins/gluster/check_cluster_vol_usage.py hostname: host1.example.com kernel: 2.6.32-573.22.1.el6.x86_64 last_occurrence: 1467059212 machineid: sosreport_uploader-dmidecode=bd5f7ccc07cb4ade159083aac7a9611bc8d9588e5e7672b6908d8771ef49ddb3 pkg_arch: noarch pkg_epoch: 0 pkg_name: nagios-server-addons pkg_release: 1.el6rhs pkg_version: 0.2.3 time: Mon 20 Jun 2016 10:42:52 AM ACST uid: 5076 username: nagios sosreport.tar.xz: Binary file, 1569292 bytes backtrace: :decoder.py:177:JSONObject:ValueError: end is out of bounds : :Traceback (most recent call last): : File "/usr/lib64/nagios/plugins/gluster/check_cluster_vol_usage.py", line 93, in <module> : numVolumes, used, avail = checkVolumePerfData(args.hostgroup) : File "/usr/lib64/nagios/plugins/gluster/check_cluster_vol_usage.py", line 49, in checkVolumePerfData : perf_data = json.loads(perf_data_output) : File "/usr/lib64/python2.6/json/__init__.py", line 307, in loads : return _default_decoder.decode(s) : File "/usr/lib64/python2.6/json/decoder.py", line 319, in decode : obj, end = self.raw_decode(s, idx=_w(s, 0).end()) : File "/usr/lib64/python2.6/json/decoder.py", line 336, in raw_decode : obj, end = self._scanner.iterscan(s, **kw).next() : File "/usr/lib64/python2.6/json/scanner.py", line 55, in iterscan : rval, next_pos = action(m, context) : File "/usr/lib64/python2.6/json/decoder.py", line 217, in JSONArray : value, end = iterscan(s, idx=end, context=context).next() : File "/usr/lib64/python2.6/json/scanner.py", line 55, in iterscan : rval, next_pos = action(m, context) : File "/usr/lib64/python2.6/json/decoder.py", line 217, in JSONArray : value, end = iterscan(s, idx=end, context=context).next() : File "/usr/lib64/python2.6/json/scanner.py", line 55, in iterscan : rval, next_pos = action(m, context) : File "/usr/lib64/python2.6/json/decoder.py", line 177, in JSONObject : key, end = scanstring(s, end, encoding, strict) :ValueError: end is out of bounds : :Local variables in innermost frame: :iterscan: <bound method Scanner.iterscan of <json.scanner.Scanner object at 0x92e1d0>> :pairs: {u'VOL_NAME': u'vol42', u'_GENERATED_BY_AUTOCONFIG': u'1'} :end: 8192 :context: <json.decoder.JSONDecoder object at 0x92e7d0> :encoding: None :_w: <built-in method match of _sre.SRE_Pattern object at 0x9300b8> :value: u'1' :strict: True :s: '[["Volume Utilization - vol1","Default","utilization=0.62%;70;90 total=32210167808.00 used=200765516.00 free=32009402292.00",{"VOL_NAME":"vol1","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol2","Default","utilization=0.00%;70;90 total=104857600.00 used=0.00 free=104857600.00",{"VOL_NAME":"vol2","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol3","Default","utilization=2.70%;70;90 total=104857600.00 used=2832924.00 free=102024676.00",{"VOL_NAME":"vol3","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol4","Default","utilization=0.00%;70;90 total=104857600.00 used=240.00 free=104857360.00",{"VOL_NAME":"vol4","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol5","Default","utilization=1.32%;70;90 total=104857600.00 used=1386236.00 free=103471364.00",{"VOL_NAME":"vol5","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol6","Default","utilization=0.08%;70;90 total=52395008.00 used=42068.00 free=52352940.00",{"VOL_NAME":"vol6","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol7","Default","utilization=0.06%;70;90 total=52395008.00 used=33604.00 free=52361404.00",{"VOL_NAME":"vol7","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol8","Default","utilization=0.00%;70;90 total=52428800.00 used=0.00 free=52428800.00",{"VOL_NAME":"vol8","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol9","Default","utilization=0.00%;70;90 total=1048576.00 used=0.00 free=1048576.00",{"VOL_NAME":"vol9","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol10","Default","",{"VOL_NAME":"vol10","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol11","Default","utilization=0.00%;70;90 total=1073741824.00 used=0.00 free=1073741824.00",{"VOL_NAME":"vol11","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol12","Default","",{"VOL_NAME":"vol12","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol13","Default","utilization=56.71%;70;90 total=10735331328.00 used=6087551236.00 free=4647780092.00",{"VOL_NAME":"vol13","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol14","Default","utilization=0.00%;70;90 total=524288000.00 used=16.00 free=524287984.00",{"VOL_NAME":"vol14","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol15","Default","",{"VOL_NAME":"vol15","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol16","Default","",{"VOL_NAME":"vol16","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol17","Default","utilization=1.10%;70;90 total=52428800.00 used=574384.00 free=51854416.00",{"VOL_NAME":"vol17","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol18","Default","utilization=0.00%;70;90 total=52428800.00 used=0.00 free=52428800.00",{"VOL_NAME":"vol18","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol19","Default","utilization=0.62%;70;90 total=32210167808.00 used=199485444.00 free=32010682364.00",{"VOL_NAME":"vol19","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol20","Default","",{"VOL_NAME":"vol20","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol21","Default","utilization=3.51%;70;90 total=15728640.00 used=551960.00 free=15176680.00",{"VOL_NAME":"vol21","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol22","Default","utilization=62.60%;70;90 total=209715200.00 used=131289864.00 free=78425336.00",{"VOL_NAME":"vol22","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol23","Default","utilization=0.32%;70;90 total=10467328.00 used=33612.00 free=10433716.00",{"VOL_NAME":"vol23","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol24","Default","utilization=0.32%;70;90 total=10467328.00 used=33988.00 free=10433340.00",{"VOL_NAME":"vol24","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol25","Default","",{"VOL_NAME":"vol25","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol26","Default","utilization=0.00%;70;90 total=10485760.00 used=0.00 free=10485760.00",{"VOL_NAME":"vol26","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol27","Default","utilization=27.89%;70;90 total=104857600.00 used=29247488.00 free=75610112.00",{"VOL_NAME":"vol27","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol28","Default","utilization=39.96%;70;90 total=6979321856.00 used=2788936528.00 free=4190385328.00",{"VOL_NAME":"vol28","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol29","Default","utilization=0.00%;70;90 total=104857600.00 used=0.00 free=104857600.00",{"VOL_NAME":"vol29","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol30","Default","utilization=0.00%;70;90 total=10485760.00 used=0.00 free=10485760.00",{"VOL_NAME":"vol30","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol31","Default","",{"VOL_NAME":"vol31","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol32","Default","",{"VOL_NAME":"vol32","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol33","Default","utilization=0.64%;70;90 total=32210167808.00 used=204863360.00 free=32005304448.00",{"VOL_NAME":"vol33","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol34","Default","utilization=0.64%;70;90 total=32210167808.00 used=204753548.00 free=32005414260.00",{"VOL_NAME":"vol34","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol35","Default","utilization=1.61%;70;90 total=2082816.00 used=33504.00 free=2049312.00",{"VOL_NAME":"vol35","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol36","Default","utilization=0.36%;70;90 total=20953088.00 used=75048.00 free=20878040.00",{"VOL_NAME":"vol36","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol37","Default","utilization=1.15%;70;90 total=10485760.00 used=120284.00 free=10365476.00",{"VOL_NAME":"vol37","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol38","Default","utilization=0.00%;70;90 total=52428800.00 used=0.00 free=52428800.00",{"VOL_NAME":"vol38","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol39","Default","utilization=40.59%;70;90 total=52428800.00 used=21280780.00 free=31148020.00",{"VOL_NAME":"vol39","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol40","Default","",{"VOL_NAME":"vol40","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol41","Default","utilization=5.37%;70;90 total=255232.00 used=13696.00 free=241536.00",{"VOL_NAME":"vol41","_GENERATED_BY_AUTOCONFIG":"1","GLUSTER_ENTITY":"Service"}],\n["Volume Utilization - vol42","Default","",{"VOL_NAME":"call_manager","_GENERATED_BY_AUTOCONFIG":"1","' :key: u'_GENERATED_BY_AUTOCONFIG' :nextchar: '"' :match: <_sre.SRE_Match object at 0xa30a40> Version-Release number of selected component (if applicable): gluster-nagios-common-0.2.3-1.el6rhs How reproducible: Happens periodically Steps to Reproduce: See description
Steps to reproduce =================== 1. Have a cluster with Nagios set up and create 50 volumes of very long names. ( I created 50 volumes of the name 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaavol$i' where $i=(1..50)) 2. Run the command '/usr/lib64/nagios/plugins/gluster/check_cluster_vol_usage.py' and check the output. When all is well, it prints 'OK' else shows a traceback Reproduced the issue in nagios-server-addons-0.2.5-1.el7rhgs.x86_64. Hit the traceback, and was able to see the ABRT message in /var/log/messages. Updated the package to nagios-server-addons-0.2.6-1.el7rhgs.x86_64 in the same setup. Did not see any traceback. /var/log/messages showed a clean log. Moving this BZ to fixed in 3.2. Detailed logs are pasted below. nagios-server-addons-0.2.5-1 ============================== /var/log/messages/ snippet 13793 Nov 21 17:13:30 dhcp46-239 python: detected unhandled Python exception in '/usr/lib64/nagios/plugins/gluster/check_cluster_vol_usage.py' 13794 Nov 21 17:13:35 dhcp46-239 python: communication with ABRT daemon failed: timed out 13795 Nov 21 17:14:01 dhcp46-239 systemd: Started Session 13397 of user root. 13796 Nov 21 17:14:01 dhcp46-239 systemd: Starting Session 13397 of user root. [root@dhcp46-239 ~]# /usr/lib64/nagios/plugins/gluster/check_cluster_vol_usage.py -hg rhel7_nagios Traceback (most recent call last): File "/usr/lib64/nagios/plugins/gluster/check_cluster_vol_usage.py", line 93, in <module> numVolumes, used, avail = checkVolumePerfData(args.hostgroup) File "/usr/lib64/nagios/plugins/gluster/check_cluster_vol_usage.py", line 49, in checkVolumePerfData perf_data = json.loads(perf_data_output) File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads return _default_decoder.decode(s) File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib64/python2.7/json/decoder.py", line 382, in raw_decode obj, end = self.scan_once(s, idx) ValueError: Unterminated string starting at: line 36 column 2 (char 8145) [root@dhcp46-239 ~]# nagios-server-addons-0.2.6-1 ============================== [root@dhcp46-239 ~]# /usr/lib64/nagios/plugins/gluster/check_cluster_vol_usage.py -hg rhel7_nagios OK - used 2% of available 1385.17285156 GB|used=2;70;95;0;100; [root@dhcp46-239 ~]# [root@dhcp46-239 ~]#
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2017-0492.html