Bug 955902 - /usr/bin/munin-cron: not a reference at /usr/share/perl5/vendor_perl/Munin/Master/Utils.pm line 866
Summary: /usr/bin/munin-cron: not a reference at /usr/share/perl5/vendor_perl/Munin/Ma...
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: munin
Version: 20
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: d. johnson
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Keywords: Reopened
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-04-24 03:26 UTC by Edgar Hoch
Modified: 2015-03-07 04:57 UTC (History)
15 users (show)

(edit)
Clone Of:
(edit)
Last Closed: 2015-03-07 04:57:12 UTC


Attachments (Terms of Use)

Description Edgar Hoch 2013-04-24 03:26:57 UTC
Description of problem:

After installing package "munin" cron sends a mail to root every five minutes with subject "Cron <munin@rotschulterbussard> test -x /usr/bin/munin-cron && /usr/bin/munin-cron" and the following content line:

not a reference at /usr/share/perl5/vendor_perl/Munin/Master/Utils.pm line 950.


Version-Release number of selected component (if applicable):
munin-2.0.12-2.fc18.noarch

How reproducible:
Always.

Steps to Reproduce:
1. Install munin.
2. Check the mail of user root after at most five minutes.
  
Actual results:
Error message (see above).

Expected results:
No regular cron mail from munin.

Additional info:
Bug 867810 seems to describe a similar problem of Fedora 16, but it was not fixed.

Comment 1 d. johnson 2013-04-24 13:19:49 UTC
Can you provide the output of:

ls -ld /var/lib/munin /var/lib/munin/plugin-state /var/lib/munin/plugin-state/nobody

rpm -qa munin\*



Then try this if the last one does not exist?

sudo install -d -m 0755 -o nobody -g root /var/lib/munin/plugin-state/nobody

Comment 2 Gilles Fauvie | OPENDBTEAM 2013-04-24 13:41:15 UTC
Hello,

Same problem on a RHEL 6.4.

The outputs you ask are:

[root@DBSP_H5_MigV data]# ls -ld /var/lib/munin /var/lib/munin/plugin-state /var/lib/munin/plugin-state/nobody
drwxr-xr-x 3 munin  munin 4096 Apr 24 15:35 /var/lib/munin
drwxrwxr-x 5 root   munin 4096 Mar 24 04:42 /var/lib/munin/plugin-state
drwxr-xr-x 2 nobody root  4096 Apr 24 14:30 /var/lib/munin/plugin-state/nobody

[root@DBSP_H5_MigV data]# rpm -qa munin\*
munin-2.0.12-2.el6.noarch
munin-node-2.0.12-2.el6.noarch
munin-common-2.0.12-2.el6.noarch

The command you gave (install -d -m 0755 -o nobody -g root /var/lib/munin/plugin-state/nobody) seems not fixing the problem.

Best Regards,

Comment 3 Gilles Fauvie | OPENDBTEAM 2013-04-24 14:39:37 UTC
Output of a manual launch of munin in debug mode:

-bash-4.1$ /usr/bin/munin-cron --debug
2013/04/24 16:31:42 [DEBUG] Creating new lock file /var/run/munin/munin-update.lock
2013/04/24 16:31:42 [DEBUG] Creating lock : /var/run/munin/munin-update.lock succeeded
2013/04/24 16:31:42 [INFO]: Starting munin-update
2013/04/24 16:31:42 [DEBUG] Starting work
2013/04/24 16:31:42 [DEBUG] Active workers: 0/16
2013/04/24 16:31:42 [DEBUG] Active workers: 1/16
2013/04/24 16:31:42 [DEBUG] Starting Munin::Master::UpdateWorker<localhost;localhost>
2013/04/24 16:31:42 [DEBUG] Creating new lock file /var/run/munin/munin-localhost-localhost.lock
2013/04/24 16:31:42 [DEBUG] Creating lock : /var/run/munin/munin-localhost-localhost.lock succeeded
2013/04/24 16:31:42 [DEBUG] Reading state for localhost-localhost in /var/lib/munin/state-localhost-localhost.storable
2013/04/24 16:31:42 [INFO] starting work in 1310 for localhost/127.0.0.1:4949.
2013/04/24 16:31:42 TLS set to "disabled".
2013/04/24 16:31:42 [DEBUG] Negotiating capabilities
2013/04/24 16:31:42 [DEBUG] Writing to socket: "cap multigraph dirtyconfig
".
2013/04/24 16:31:42 [DEBUG] Node says /cap multigraph dirtyconfig/
2013/04/24 16:31:42 [DEBUG] Writing to socket: "list DBSP_H5_MigV
".
2013/04/24 16:31:42 [WARNING] Config node localhost listed no services for DBSP_H5_MigV.  Please see http://munin-monitoring.org/wiki/FAQ_no_graphs for further information.
2013/04/24 16:31:42 [DEBUG] Writing to socket: "quit
".
2013/04/24 16:31:42 [DEBUG] quit: 0.000204 sec on localhost/127.0.0.1/4949
2013/04/24 16:31:42 [DEBUG] Everything went smoothly.
2013/04/24 16:31:42 [DEBUG] Writing state for localhost-localhost in /var/lib/munin/state-localhost-localhost.storable
2013/04/24 16:31:42 [DEBUG] Exit status 0 for worker Munin::Master::UpdateWorker<localhost;localhost>
2013/04/24 16:31:42 [INFO]: Munin-update finished for node localhost;localhost (0.01 sec)
2013/04/24 16:31:42 [INFO] Reaping Munin::Master::UpdateWorker<localhost;localhost>.  Exit value/signal: 0/0
2013/04/24 16:31:42 [DEBUG] Work done
2013/04/24 16:31:42 [DEBUG] Creating new lock file /var/run/munin/munin-datafile.lock
2013/04/24 16:31:42 [DEBUG] Creating lock : /var/run/munin/munin-datafile.lock succeeded
2013/04/24 16:31:42 [DEBUG] Writing state to /var/lib/munin/datafile.storable
2013/04/24 16:31:42 [INFO]: Munin-update finished (0.02 sec)
2013/04/24 16:31:42 [INFO] Starting munin-limits, getting lock /var/run/munin/munin-limits.lock
2013/04/24 16:31:42 [DEBUG] Creating new lock file /var/run/munin/munin-limits.lock
2013/04/24 16:31:42 [DEBUG] Creating lock : /var/run/munin/munin-limits.lock succeeded
2013/04/24 16:31:42 [DEBUG] munin_readconfig: found Storable version of /var/lib/munin/limits.storable, using it
2013/04/24 16:31:42 [DEBUG] Set default "contacts" to ""
2013/04/24 16:31:42 [DEBUG] Writing state to /var/lib/munin/limits
2013/04/24 16:31:42 [DEBUG] Closing filehandle "/var/lib/munin/limits"...
2013/04/24 16:31:42 [DEBUG] Writing state to /var/lib/munin/limits.storable
2013/04/24 16:31:42 [INFO] munin-limits finished (0.00 sec)
2013/04/24 16:31:43 [DEBUG] Determined that cgiurl_graph is /munin-cgi/munin-cgi-graph
2013/04/24 16:31:43 [DEBUG] munin_readconfig: found Storable version of /var/lib/munin/limits.storable, using it
2013/04/24 16:31:43 [DEBUG] Writing state to /var/lib/munin/htmlconf.storable
2013/04/24 16:31:43 [PERL WARNING] Use of uninitialized value in list assignment at /usr/share/perl5/vendor_perl/Munin/Master/Utils.pm line 134.
2013/04/24 16:31:43 [PERL WARNING] data is null at  at /usr/share/perl5/vendor_perl/Munin/Master/Utils.pm line 947
	Munin::Master::Utils::munin_write_storable('/var/lib/munin/htmlconf.storable', undef) called at /usr/share/perl5/vendor_perl/Munin/Master/Utils.pm line 964
	Munin::Master::Utils::munin_writeconfig_storable('/var/lib/munin/htmlconf.storable', undef) called at /usr/share/perl5/vendor_perl/Munin/Master/HTMLOld.pm line 181
	Munin::Master::HTMLOld::get_config(0) called at /usr/share/perl5/vendor_perl/Munin/Master/HTMLOld.pm line 191
	Munin::Master::HTMLOld::html_main() called at /usr/share/munin/munin-html line 40
not a reference at /usr/share/perl5/vendor_perl/Munin/Master/Utils.pm line 950

Hope I will help you.

Comment 4 d. johnson 2013-04-24 15:41:17 UTC
Not sure; I cannot reproduce the issue you have above:

munin-2.0.12-2.fc18.noarch munin-node-2.0.12-2.fc18.noarch munin-common-2.0.12-2.fc18.noarch munin-async-2.0.12-2.fc18.noarch

munin-2.0.12-2.fc17.noarch munin-async-2.0.12-2.fc17.noarch munin-node-2.0.12-2.fc17.noarch munin-common-2.0.12-2.fc17.noarch

munin-common-2.0.12-2.el6.noarch munin-2.0.12-2.el6.noarch munin-node-2.0.12-2.el6.noarch munin-async-2.0.12-2.el6.noarch munin-java-plugins-2.0.12-2.el6.noarch

munin-common-2.0.12-2.el5 munin-java-plugins-2.0.12-2.el5 munin-ruby-plugins-2.0.12-2.el5 munin-node-2.0.12-2.el5 munin-cgi-2.0.12-2.el5 munin-2.0.12-2.el5 munin-async-2.0.12-2.el5

Most recent I can find is from Date: Fri, 25 Jan 2013 16:59:38 -0700  (Shortly after 2.0.11-3)

This may take a while to sort out.

Comment 5 Edgar Hoch 2013-04-24 17:18:56 UTC
(In reply to comment #1)
> Can you provide the output of:
> 
> ls -ld /var/lib/munin /var/lib/munin/plugin-state
> /var/lib/munin/plugin-state/nobody

drwxr-xr-x. 3 munin  munin 4096 24. Apr 05:17 /var/lib/munin
drwxrwxr-x. 5 root   munin 4096 24. Mär 04:42 /var/lib/munin/plugin-state
drwxr-xr-x. 2 nobody root  4096 24. Apr 00:18 /var/lib/munin/plugin-state/nobody

> rpm -qa munin\*

munin-node-2.0.12-2.fc18.noarch
munin-common-2.0.12-2.fc18.noarch
munin-2.0.12-2.fc18.noarch

> Then try this if the last one does not exist?
> 
> sudo install -d -m 0755 -o nobody -g root /var/lib/munin/plugin-state/nobody

The directory exists already.

I did a fresh install of Fedora 18 on a virtual machine, with no modifications regarding munin.

Comment 6 Edgar Hoch 2013-05-09 21:35:42 UTC
yum-updatesd have done an update to

munin-common-2.0.13-1.fc18.noarch
munin-node-2.0.13-1.fc18.noarch
munin-2.0.13-1.fc18.noarch

Now I get cron messages every five minutes with mail subject

"Cron <munin@rauhfussbussard> test -x /usr/bin/munin-cron && /usr/bin/munin-cron"

and mail content

not a reference at /usr/share/perl5/vendor_perl/Munin/Master/Utils.pm line 866.


I don't know if one reason for the errer may be that I didn't change the configuration of munin. The munin packages are just installed, without a change.

When a package is installed, it should not create errors. I think, it should do nothing, it should not run cron without explicit configuration by the user (it is possible to check for a flag set by the user or set / changed by a command (like munin-activate or eventually with systemctl?).


Here a debug output:

munin$ /usr/bin/munin-cron --debug
2013/05/09 22:34:03 [DEBUG] Creating new lock file /var/run/munin/munin-update.lock
2013/05/09 22:34:03 [DEBUG] Creating lock : /var/run/munin/munin-update.lock succeeded
2013/05/09 22:34:03 [INFO]: Starting munin-update
2013/05/09 22:34:03 [DEBUG] Starting work
2013/05/09 22:34:03 [DEBUG] Active workers: 0/16
2013/05/09 22:34:03 [DEBUG] Active workers: 1/16
2013/05/09 22:34:03 [DEBUG] Starting Munin::Master::UpdateWorker<localhost;localhost>
2013/05/09 22:34:03 [DEBUG] Lock /var/run/munin/munin-localhost-localhost.lock already exists, checking process
2013/05/09 22:34:03 [DEBUG] Lock contained pid '26412'
2013/05/09 22:34:03 [INFO] Process 26412 is dead, stealing lock, removing file
2013/05/09 22:34:03 [DEBUG] Creating new lock file /var/run/munin/munin-localhost-localhost.lock
2013/05/09 22:34:03 [DEBUG] Creating lock : /var/run/munin/munin-localhost-localhost.lock succeeded
2013/05/09 22:34:03 [DEBUG] Reading state for localhost-localhost in /var/lib/munin/state-localhost-localhost.storable
2013/05/09 22:34:03 [INFO] starting work in 26657 for localhost/127.0.0.1:4949.
2013/05/09 22:34:03 [FATAL] Socket read from localhost failed.  Terminating process. at /usr/share/perl5/vendor_perl/Munin/Master/UpdateWorker.pm line 254.
2013/05/09 22:34:03 [ERROR] Munin::Master::UpdateWorker<localhost;localhost> died with '[FATAL] Socket read from localhost failed.  Terminating process. at /usr/share/perl5/vendor_perl/Munin/Master/UpdateWorker.pm line 254.
'
2013/05/09 22:34:03 [DEBUG] Exit status 18 for worker Munin::Master::UpdateWorker<localhost;localhost>
2013/05/09 22:34:13 [INFO] Remaining workers: localhost;localhost
2013/05/09 22:34:13 [DEBUG] Got error from 26657: 4608.  Handling error
2013/05/09 22:34:13 [DEBUG] In exception handler for failed worker localhost;localhost
2013/05/09 22:34:13 [INFO] Reaping Munin::Master::UpdateWorker<localhost;localhost>.  Exit value/signal: 18/0
2013/05/09 22:34:13 [DEBUG] Active workers: 0/16
2013/05/09 22:34:13 [DEBUG] Work done
2013/05/09 22:34:13 [DEBUG] Creating new lock file /var/run/munin/munin-datafile.lock
2013/05/09 22:34:13 [DEBUG] Creating lock : /var/run/munin/munin-datafile.lock succeeded
2013/05/09 22:34:13 [INFO] No old data available for failed worker localhost;localhost.  This node will disappear from the html web page hierarchy
2013/05/09 22:34:13 [DEBUG] Writing state to /var/lib/munin/datafile.storable
2013/05/09 22:34:13 [INFO]: Munin-update finished (10.01 sec)
2013/05/09 22:34:13 [INFO] Starting munin-limits, getting lock /var/run/munin/munin-limits.lock
2013/05/09 22:34:13 [DEBUG] Creating new lock file /var/run/munin/munin-limits.lock
2013/05/09 22:34:13 [DEBUG] Creating lock : /var/run/munin/munin-limits.lock succeeded
2013/05/09 22:34:13 [DEBUG] munin_readconfig: found Storable version of /var/lib/munin/limits.storable, using it
2013/05/09 22:34:13 [DEBUG] Set default "contacts" to ""
2013/05/09 22:34:13 [DEBUG] Writing state to /var/lib/munin/limits
2013/05/09 22:34:13 [DEBUG] Closing filehandle "/var/lib/munin/limits"...
2013/05/09 22:34:13 [DEBUG] Writing state to /var/lib/munin/limits.storable
2013/05/09 22:34:13 [INFO] munin-limits finished (0.00 sec)
2013/05/09 22:34:13 [DEBUG] munin_readconfig: found Storable version of /var/lib/munin/limits.storable, using it
2013/05/09 22:34:13 [DEBUG] Writing state to /var/lib/munin/htmlconf.storable
2013/05/09 22:34:13 [PERL WARNING] data is null at  at /usr/share/perl5/vendor_perl/Munin/Master/Utils.pm line 863.
	Munin::Master::Utils::munin_write_storable('/var/lib/munin/htmlconf.storable', undef) called at /usr/share/perl5/vendor_perl/Munin/Master/Utils.pm line 877
	Munin::Master::Utils::munin_writeconfig_storable('/var/lib/munin/htmlconf.storable', undef) called at /usr/share/perl5/vendor_perl/Munin/Master/HTMLOld.pm line 182
	Munin::Master::HTMLOld::get_config(0) called at /usr/share/perl5/vendor_perl/Munin/Master/HTMLOld.pm line 192
	Munin::Master::HTMLOld::html_main() called at /usr/share/munin/munin-html line 40
not a reference at /usr/share/perl5/vendor_perl/Munin/Master/Utils.pm line 866.

Comment 7 Gilles Fauvie | OPENDBTEAM 2013-05-13 15:17:45 UTC
Hello,

Ok, I think that I solved the problem.

In my case (munin 2.0.13 via EPEL) on RHEL 6.4, the origin of the problem was "_" (underscore) in the hostname. Munin uses underscore to pass parameters to the plugins.

Try to change your hostname (and check munin-node.conf and munin.conf). It should work. If yes, I think that the status can be change to "solved".

Best Regards,

Gilles Fauvie - opendbteam.com

Comment 8 Fedora Admin XMLRPC Client 2013-05-19 19:52:27 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 9 Mehmet Avcioglu 2013-05-29 13:34:12 UTC
On a fresh install of Red Hat 6.4 using munin 2.0.12-2 or 2.0.14-1 from EPEL Testing still getting the same errors on root mail except line number changed to 866 from 950. I don't have an "_" on my hostname.

I believe the cause is munin-node is not running. I did the following which fixed the error message.

$ rm -f /var/lib/munin/{datafile,datafile.storable,htmlconf.storable}
$ /etc/init.d/munin-node start

Not sure if the first command is necessary. After running once, stoping munin-node did not cause any further error messages.

Comment 10 Anonymous account 2013-06-07 13:48:56 UTC
Hello,

And in my case (munin-2.0.14-1.el6.noarch via EPEL) on CentOS 6.0, the origin of the problem was "_" (underscore) in the hostname. After change hostname, munin work.

Comment 11 Edgar Hoch 2013-06-07 13:57:19 UTC
We have no "_" in the hostname, but a "-" in the domainname. I don't know if this the origin of the problem, but we cannot change the domainname. "-" is a common character in domainnames (and hostnames), so this must not be a problem.

Comment 12 d. johnson 2013-06-14 01:11:59 UTC
If you are still having this problem, can you try something?

in /etc/munin/munin-node.conf, can you add a line like "host_name localhost" (with the name you have in /etc/hosts if the node is not on the master)

Comment 13 Till Maas 2013-07-10 11:08:13 UTC
Here the bug still appears on a fresh F19 install  on  a system with a dash in the hostname and "host_name whatever" in munin-node.conf does not seem to help.

Is there an upstream bug report about this?

Comment 14 Gabriele Giacone 2013-07-21 14:28:10 UTC
Worked around with http://munin-monitoring.org/ticket/1285 by hardcoding localhost.
munin 2.0.16-1 on centos 5.9

Comment 15 Petr Novak 2013-12-03 18:26:13 UTC
I had a same problem related to address 127.0.0.1 in my host tree in my /etc/munin/munin.conf. I switched first two lines in /etc/hosts, so 127.0.0.1 is now before ::1, and it solved my issue. (CentOS 6.5)

Comment 16 Fedora End Of Life 2013-12-21 13:12:52 UTC
This message is a reminder that Fedora 18 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 18. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '18'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 18's end of life.

Thank you for reporting this issue and we are sorry that we may not be 
able to fix it before Fedora 18 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior to Fedora 18's end of life.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 17 Håkon Løvdal 2013-12-31 11:40:13 UTC
Do not close this bug just because it was created for Fedora 18, I got this problem (emails every 5 minutes with "not a reference at /usr/share/perl5/vendor_perl/Munin/Master/Utils.pm line 866") on a freshly installed Fedora 20 system.

Munin is installed but not configured by me in any way. There is no non-alphabet characters in the hostname.

$ ls -ld /var/lib/munin /var/lib/munin/plugin-state /var/lib/munin/plugin-state/nobody
drwxr-xr-x. 4 munin  munin 4096 des.  31 12:30 /var/lib/munin
drwxrwxr-x. 5 root   munin 4096 des.   8 08:28 /var/lib/munin/plugin-state
drwxr-xr-x. 2 nobody root  4096 des.  29 19:09 /var/lib/munin/plugin-state/nobody
You have new mail in /home/hlovdal/Maildir
$ rpm -qa munin\*
munin-common-2.0.19-1.fc20.noarch
munin-node-2.0.19-1.fc20.noarch
munin-2.0.19-1.fc20.noarch
$

Comment 18 d. johnson 2014-01-01 01:52:48 UTC
What do you have in your config file for host_name ?

If you look in your /etc/hosts file, do you see an exact match for host_name (above) ?

Comment 19 d. johnson 2014-01-03 01:02:33 UTC
After installing, set host_name to something found in /etc/hosts and restart munin-node.  tell me if the problem continues.  so far, every case i've seen is resolved by this, including fresh installs.

The dbus / NM / systemd magic isnt working for hostname, so setting it to $HOSTNAME results in the error you see.

Using something listed in /etc/hosts ensures you do not see this message.

Comment 20 Edgar Hoch 2014-01-12 02:50:49 UTC
Thanks for your answer.

munin-2.0.19-1.fc20.noarch still sends an error mail to root every five minutes:

Subject: Cron <munin@rauhfussbussard> test -x /usr/bin/munin-cron && /usr/bin/munin-cron
Mail body: not a reference at /usr/share/perl5/vendor_perl/Munin/Master/Utils.pm line 866.


I have not modified any of the munin files. I have just installed the packages.

The answer from "hostname" and from "uname -n" does not match any entry in /etc/hosts. I don't want to enter the hostname in /etc/hosts because we use dns and nis or ldap. The (default) two lines for localhost are the only ones which are in every /etc/hosts.

Doesn't munin check the hostname depending on /etc/nsswitch.conf (e.g. use system functions to resolve the hostname)??? Why do you think that the hostname must match an entry in /etc/hosts?



I think munin should do NOTHING if I just install the packages but not configure or enable it. It SHOULD NOT even run cron jobs every five minutes!

I just want to install the same list of packages on all computers independed of using it on a host. If I want to use it, I have to configure and enable it.

Either the cron job should not be installed, or the cron job script of munin should check something (e.g. a variable in a config file) if munin is enabled or not, und do more work only if it es enabled.


I currently don't have munin running, but I want to configure it for some hosts some time. At the moment I don't have the time to configure munin (maybe probably in the next weeks or months), but I think if enabled it should do some default jobs for the local host (using the hostname returned from "hostname" or "uname -n", resolved using the getent* or similar functions).

Comment 21 d. johnson 2014-01-12 05:51:44 UTC
> The answer from "hostname" and from "uname -n" does not match any entry in /etc/hosts.


^^ That is why you get the warning every 5mins.

See comment 19 for why.  This error is not a munin bug at all. This is related to the way your NSS config is setup.

Comment 22 Edgar Hoch 2014-01-12 06:29:33 UTC
It is an error if a package needs manual intervention to prevent flooding the mailbox with error mails. Package installation must run unattended and work error free out of the box.

It is an error if a package depends on an matching entry in /etc/hosts when nearly all other packages have no problem with his configuration. It seems to me that munin uses an improper methode. It must not crash because an entry in /etc/hosts is missing.

It is an error if perl crashes. The error message is a perl error message in a munin perl program - it is obvious that this is an error of munin. A program must not crash because of predictable situations - it has to handle this situation!


For munin:

"not a reference at /usr/share/perl5/vendor_perl/Munin/Master/Utils.pm line 866." is a perl error message in a perl program of munin. So this is a munin error.

I can understand that munin does not crash when there is a matching entry in /etc/hosts. But it is not my fault that munin crashes when this condition is not fulfilled - it's the problem of munin! Don't give me the fault of munin's program code!

Comment 23 Edgar Hoch 2014-01-12 09:24:17 UTC
(In reply to d. johnson from comment #21)
> See comment 19 for why.  This error is not a munin bug at all. This is
> related to the way your NSS config is setup.

You require that users / administrators should add the hostname (contained in /etc/hostname) to /etc/hosts. But his is a very bad requirement because it may break networking in some configurations / scenarios.

Consider the usual case where the network address is assiged dynamically using dhcp. /etc/hosts requires an ip address and a hostname. But with dhcp the ip address is not fixed - it may change on each boot. You cannot enter the right ip address to /etc/hosts - it may not match the assigned ip address of the network interface.

Setting network configuration to static assignment of ip address and other network parameters would solve this problem - but this would require that munin mist only be installed on hosts with static address assignment? This could not be a requirement that you _really_ think of!


Please note that the default of Fedora is that /etc/hosts only contains two entries for localhost (one for ipv4, one for ipv6), but not an entry for the hostname. An entry for the hostname was used long time ago, I am not sure, it may be until Fedora 10 oder Fedora 12. Then Fedora has changed the installation to use full qualified hostnames (hostnames with dns domain) and to not add it to /etc/hosts by default.


Please note another problem: Suppose /etc/hostname contains "myhost.example.com". Think of a scenario where the assigned ip address is changed in the dns server. If the hostname would be in /etc/hosts too, then the host may use a wrong ip address if /etc/hosts is not changed at the same time (there may be different administrators) even if the host uses dhcp. But dhcp and dns was designed for central managing of network addresses, and to avoid that each host must be changed by hand too.


Munin must be configured to not create any errors after a package installation or update. Munin must be compatible with the defaults of Fedora! It is not true that Fedora must be changed to satisfy the requirements of munin.

The munin package is in an inacceptable state because the program crashes and floods the mailbox of the administrator. Nobody can install the munin package without doing something to avoid the error mails - otherwise the mailbox or the partition containing it may be filled over time until all space is exhausted. And most of them have to search for instructions how to do it... - and they will be annoyed about this package that makes trouble. Not every user and administrator writes a bug report...

Comment 24 d. johnson 2014-01-12 14:10:04 UTC
This is a bug in the way your system is configured for NSS. It is not responding correctly to the hostname queries.

Comment 25 Till Maas 2014-01-12 15:14:58 UTC
(In reply to d. johnson from comment #24)
> This is a bug in the way your system is configured for NSS. It is not
> responding correctly to the hostname queries.

Can you please specify what is wrong? It seems to me that you say that Fedora's defaults are wrong.

Comment 26 Håkon Løvdal 2014-01-12 18:32:51 UTC
-Doctor, my arm hurts when I do this.

-Well don't do that then!

Comment 27 d. johnson 2014-01-13 03:20:44 UTC
Till, That is exactly what I have said.  There is a service that is *supposed* to handle hostname requests, as part of systemd, that fails to respond properly.  You could write a simple testcase based on the comments above or the 3 lines of perl that exemplify this issue.

It is unique to this distro.  This problem does not occur on other distros.  Arch, Debian, Ubuntu, Gentoo to name a few.

You can see this issue by adding $HOSTNAME to /etc/hosts which circumvents the NSS problem entirely by making a manual entry.  Not ideal, but does work, everytime.

Comment 28 Edgar Hoch 2014-01-13 09:36:16 UTC
I have checked the files of munin-2.0.19-1.fc20.noarch and found the following:

1. /etc/munin/munin.conf contains the lines (unmodified from the package):

[localhost]
    address 127.0.0.1
    use_node_name yes

There are matching lines in /etc/hosts:

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

It seems that your requirement are fulfilled but the error occures.


2. I tried a modified file /etc/munin/munin.conf: I added a domain to the hostname:

[localhost.localdomain]
    address 127.0.0.1
    use_node_name yes

This hostname is also matched in /etc/hosts.
In this case the error occures too.


3. It seems to me that the default configuration provided by the installation package does not use the hostname. (???)

4. I run each of the commands in /etc/cron.d/munin and found that the error is caused by /usr/share/munin/munin-html .

5. In /usr/share/perl5/vendor_perl/Munin/Master/HTMLOld.pm function get_config (line 153) returns an undefined value ("undef").
  The reason is that line 165
"$htmlconfig = generate_config($use_cache);"
  returns an undefined value ("undef").

6. In /usr/share/perl5/vendor_perl/Munin/Master/HTMLConfig.pm function generate_config (line 34) returns an undefined value because in line 60
"$cache = get_group_tree($config);"
  returns an undefined value.

7. In /usr/share/perl5/vendor_perl/Munin/Master/HTMLConfig.pm function get_group_tree (line 137) returns a null value because in line 191
"push( @$groups, grep {defined $_} get_group_tree($child, $base.munin_get_node_name($child) . "/"));"
  the @$groups array remain empty.

8. I am not sure if functions munin_get_sorted_children (/usr/share/perl5/vendor_perl/Munin/Master/HTMLConfig.pm line 381) and munin_get_children (/usr/share/perl5/vendor_perl/Munin/Master/Utils.pm line 481) work right. They modify the data structure (the hash in the array) so that I am not sure if the references are correct.

9. In /usr/share/perl5/vendor_perl/Munin/Master/Utils.pm I am wondering about the code in function munin_parse_config (line 310) for parsing a hostname (lines 350-352).
  For example, for an entry "[localhost]" in /etc/munin/munin.conf it will create a $prefix value of "localhost;localhost:".
  For "[localhost.localdomain]" it will create "localdomain;localhost.localdomain:".
  For "[myhost.example.com]" it will create "example.com;myhost.example.com:".
  This values will be used to create a hierarchical structure in the hash (variable $config in some parts of the code). I am wondering about the structure, for example about the localhost - localhost hierarchy (and in general all hostnamewithoutdomain - hostnamewithoutdomain combination).


Until now I have not found the excact part of the perl code which causes the problem and how it can be fixed. But until now I have not found system calls that causes the error. For localhost there are matching entries in /etc/hosts.

Maybe someone can found the problematic code in munin using the list of problematic code above?

Comment 29 Edgar Hoch 2014-01-13 09:40:07 UTC
I changed the summary to match the error message of Fedora 20.

Comment 30 d. johnson 2014-01-13 13:25:22 UTC
Edgar, wrong section.

in /etc/munin/munin-node.conf, set host_name to something found in /etc/hosts and restart munin-node.

Comment 31 Edgar Hoch 2014-01-13 15:33:09 UTC
(In reply to d. johnson from comment #30)
> Edgar, wrong section.

Please tell us the reason what is wrong with my statements. Please don't repeat always the same statement without a comprehensible reason.


> in /etc/munin/munin-node.conf, set host_name to something found in
> /etc/hosts and restart munin-node.

I have tried both "host_name localhost" and "host_name localhost.localdomain" in the file /etc/munin/munin-node.conf in the line after "#host_name localhost.localdomain". But the error message still occured.
A call of (as root)
  "runuser -u munin -- perl /usr/share/munin/munin-html"
prints to the shell:
  "not a reference at /usr/share/perl5/vendor_perl/Munin/Master/Utils.pm line 866."


I still cannot see what the host_name entry has to do with the problems listed in comment #28.

Comment 32 Edgar Hoch 2014-01-13 15:53:22 UTC
I found three open munin tickets at the munin website that seems to describe the same or at least a similar problem - not on Fedora, but Debian.

http://munin-monitoring.org/ticket/1257
http://munin-monitoring.org/ticket/1219
http://munin-monitoring.org/ticket/1154

Comment 33 d. johnson 2014-01-14 03:14:22 UTC
re/ #c31 - Did you restart munin-node ?  (If you did not restart it, then the change will not take effect.)

The debian bug reports you pointed to are quite outdated with no config details.  See also, "We should have a better error message, but since the workaround is just to wait, i'm pushing it to 3.0, and lower its priority."

As for your specifics Edgar,

Please run "rpm -Va munin\*" and post the output along with your config files.

I have tested the defaults *many* times with only changing the host_name parameter and nothing else, with 100% success.

Comment 34 Edgar Hoch 2014-01-14 03:45:10 UTC
(In reply to d. johnson from comment #33)
> re/ #c31 - Did you restart munin-node ?  (If you did not restart it, then
> the change will not take effect.)

How do I restart munin-node? It was not running at any time before. I just installed the package (the same list of packages on all our hosts) - so I _can_ configure and run it on _some_ of our host!

# systemctl status munin-node
munin-node.service - Munin Node Server.
   Loaded: loaded (/usr/lib/systemd/system/munin-node.service; disabled)
   Active: inactive (dead)
     Docs: man:munin-node

The only process that was running was /usr/bin/munin-cron.
I have disabled the munin cron job (using comment sign in /etc/cron.d/munin) to avoid the every five minutes mail during the testing period.


> The debian bug reports you pointed to are quite outdated with no config
> details.  See also, "We should have a better error message, but since the
> workaround is just to wait, i'm pushing it to 3.0, and lower its priority."

Yes, they are a bit outdated, but there is no notice that the problem described there is solved. And our error message is similar - not excact the same - but of course, it _may_ be solved in the meentime but not noted in the tickets.

> As for your specifics Edgar,
> 
> Please run "rpm -Va munin\*" and post the output along with your config
> files.

[root@rauhfussbussard ~]# rpm -Va munin\*
S.5....T.  c /etc/munin/munin-node.conf
S.5....T.  c /etc/cron.d/munin
.......T.    /var/www/html/munin/static/definitions.html
.......T.    /var/www/html/munin/static/dynazoom.html
.......T.    /var/www/html/munin/static/favicon.ico
.......T.    /var/www/html/munin/static/formatdate.js
.......T.    /var/www/html/munin/static/logo-h.png
.......T.    /var/www/html/munin/static/logo.png
.......T.    /var/www/html/munin/static/querystring.js
.......T.    /var/www/html/munin/static/style-1.2.css
.......T.    /var/www/html/munin/static/style-new.css
.......T.    /var/www/html/munin/static/style.css
.......T.    /var/www/html/munin/static/zoom.js

The modification of /etc/munin/munin-node.conf is the following (for testing):
# diff /etc/munin/munin-node.conf.old /etc/munin/munin-node.conf
37a38
> hostname localhost.localdomain

The only change in /etc/cron.d/munin is the comment sign:
#*/5 * * * *     munin test -x /usr/bin/munin-cron && /usr/bin/munin-cron

The other files are unmodified, only in the state that the package installation leaves.

Comment 35 d. johnson 2014-01-14 05:14:13 UTC
You will need munin-node active and running for it to work.

Example:

$ systemctl status munin-node
munin-node.service - Munin Node Server.
   Loaded: loaded (/usr/lib/systemd/system/munin-node.service; enabled)
   Active: active (running)

Comment 36 Edgar Hoch 2014-01-14 05:29:39 UTC
(In reply to d. johnson from comment #35)
> You will need munin-node active and running for it to work.
> 
> Example:
> 
> $ systemctl status munin-node
> munin-node.service - Munin Node Server.
>    Loaded: loaded (/usr/lib/systemd/system/munin-node.service; enabled)
>    Active: active (running)

You know that I just want to install the package, not to run the software (at least not on all host, only on some of them)?

Maybe that the error does not occur when munin-node is running. But this is not the point. The problem is that the package sends error mails when the package is installed. This is a behaviour that a new installed package must not do!


But behind of that: I think it is still an error if the code does not handle null and undefined values in a proper manner (e.g., no error even if no data is available...).

Comment 37 Stefan Midjich 2014-03-11 09:39:51 UTC
Have any of you tried setting selinux to permissive mode and then running munin-cron? 

I had the same problem with reference error from munin-cron and I tried permissive mode temporarily and everything worked. 

I've already had to write my own policy module to get munin-node working with selinux and it's clear to me now that the policy included in the EPEL package does not work. 

I'm no pro at selinux but if I get my policy working I'll post it here so people who are better than me can scrutinize it.

Comment 38 d. johnson 2014-03-12 01:23:56 UTC
Stefan, if you attach the AVC's you are seeing, the correct labels can be applied and the policy can be adjusted to fix whatever is wrong with it.

The policy in EPEL should just work if things are labeled correctly.

Because this bug report does not relate to your AVC issue, you may be asked to open another ticket under the selinux-policy component.

Comment 39 customercare 2014-03-13 09:42:43 UTC
munin-node-2.0.19-1.fc20.noarch:

2014/03/13 10:26:02 [PERL WARNING] data is null at  at /usr/share/perl5/vendor_perl/Munin/Master/Utils.pm line 863.
        Munin::Master::Utils::munin_write_storable('/var/lib/munin/htmlconf.storable', undef) called at /usr/share/perl5/vendor_perl/Munin/Master/Utils.pm line 877
        Munin::Master::Utils::munin_writeconfig_storable('/var/lib/munin/htmlconf.storable', undef) called at /usr/share/perl5/vendor_perl/Munin/Master/HTMLOld.pm line 182
        Munin::Master::HTMLOld::get_config(0) called at /usr/share/perl5/vendor_perl/Munin/Master/HTMLOld.pm line 192
        Munin::Master::HTMLOld::html_main() called at /usr/share/munin/munin-html line 40 


Happens since i upgraded the system from fc19 with A WORKING munin setup. 

munin.config on the system has not changed from fc19 to fc20 .

SELINUX is not involved here at all.

Comment 40 customercare 2014-03-13 10:26:22 UTC
Bug identified:

get_group_tree(0) shall return the htmlconfig , but it doesn't and it can't:

"HTMLConfig.pm" line 203:

return unless $visible;

The callerfunction expects it to return something:

"HTMLOld.pm" [readonly] line 165:

$htmlconfig = generate_config($use_cache);
...
munin_writeconfig_storable($cachefile, $htmlconfig);

with no config in $htmlconfig, writeconfig fails. 

this happens, because there are no "children" to loop over, so $visible never gets set and the function has no chance to return usefull data.

I checked the fc19 mainserver with generates our pages and $visible was set there to "1" sometimes.

Attention:

It is possible, that the fc20 client on our server should not generate htmlpages, because our main system does that. BUT, it should not crash with a stacktrace at any time. A crash always means, there something broken.

Comment 41 d. johnson 2014-03-14 03:16:11 UTC
customercare@resellerdesktop,

in /etc/munin/munin-node.conf, set host_name to something found in /etc/hosts and restart munin-node.

Comment 42 customercare 2014-03-14 11:40:35 UTC
The bug was not caused by a wrong hostname.

Explanation: 

/etc/munin/plugins/ contains normaly symlinks to "/usr/share/munin/plugins/pluginname"

As munin was upgraded via yum in 2012 ( yes we didn't notice it, for quite some time :) ) the plugins directorystructure was switched to those symlinks. But that did not work on all active installations. Some never got the symlinks. 

The bug happens, when there are no children to work with ( see comment #40 ). 
Is can be caused by a wrong hostname in munin.conf, but it also happens, when there are no services in the plugins directory. 

FIX:  fill your plugins directory with these symlinks :

lrwxrwxrwx 1 root root   28  4. Feb 2013  cpu -> /usr/share/munin/plugins/cpu
lrwxrwxrwx 1 root root   27  4. Feb 2013  df -> /usr/share/munin/plugins/df
lrwxrwxrwx 1 root root   33  4. Feb 2013  df_inode -> /usr/share/munin/plugins/df_inode

...etc..etc..

lrwxrwxrwx 1 root root   31  4. Feb 2013  vmstat -> /usr/share/munin/plugins/vmstat    

Restart munin-node and bug is gone. 

ATTENTION:

no plugins = no services : leads to a complete munin failure, because none of the tools produces anything of value. This is a resonable bevavior if your not telling munin what to test, obviosly.

This should be fixed by a test, if there are any plugins enabled! and a correct intuitive error message, because it does not make ANY SENSE to have no services at all :)

It would be nice, if this feature request makes it to the dev of munin.

Comment 43 customercare 2014-07-25 06:21:39 UTC
Can we close this now ?

Comment 44 Edgar Hoch 2014-07-25 10:53:46 UTC
(In reply to customercare from comment #43)
> Can we close this now ?

No - because the problem still exists.

Some minutes ago I have installed munin ("yum install munin") on a Fedora 19 and a Fedora 20 machine to see what happens. Now, on both machines, root gets e-mails everv five minutes from cron with the following content:

not a reference at /usr/share/perl5/vendor_perl/Munin/Master/Utils.pm line 866.


Please note that after installation /etc/munin/plugins contains symbolic links as suggested in comment #42:

/etc/munin/plugins:
insgesamt 0
lrwxrwxrwx. 1 root root 28 25. Jul 12:43 cpu -> /usr/share/munin/plugins/cpu
lrwxrwxrwx. 1 root root 27 25. Jul 12:43 df -> /usr/share/munin/plugins/df
lrwxrwxrwx. 1 root root 33 25. Jul 12:43 df_inode -> /usr/share/munin/plugins/df_inode
lrwxrwxrwx. 1 root root 34 25. Jul 12:43 diskstats -> /usr/share/munin/plugins/diskstats
lrwxrwxrwx. 1 root root 32 25. Jul 12:43 entropy -> /usr/share/munin/plugins/entropy
lrwxrwxrwx. 1 root root 39 25. Jul 12:43 exim_mailqueue -> /usr/share/munin/plugins/exim_mailqueue
lrwxrwxrwx. 1 root root 30 25. Jul 12:43 forks -> /usr/share/munin/plugins/forks
lrwxrwxrwx. 1 root root 37 25. Jul 12:43 fw_conntrack -> /usr/share/munin/plugins/fw_conntrack
lrwxrwxrwx. 1 root root 43 25. Jul 12:43 fw_forwarded_local -> /usr/share/munin/plugins/fw_forwarded_local
lrwxrwxrwx. 1 root root 35 25. Jul 12:43 fw_packets -> /usr/share/munin/plugins/fw_packets
lrwxrwxrwx. 1 root root 41 25. Jul 12:43 hddtemp_smartctl -> /usr/share/munin/plugins/hddtemp_smartctl
lrwxrwxrwx. 1 root root 28 25. Jul 12:43 if_brnetboot -> /usr/share/munin/plugins/if_
lrwxrwxrwx. 1 root root 32 25. Jul 12:43 if_err_brnetboot -> /usr/share/munin/plugins/if_err_
lrwxrwxrwx. 1 root root 32 25. Jul 12:43 if_err_netboot -> /usr/share/munin/plugins/if_err_
lrwxrwxrwx. 1 root root 32 25. Jul 12:43 if_err_virbr0 -> /usr/share/munin/plugins/if_err_
lrwxrwxrwx. 1 root root 32 25. Jul 12:43 if_err_virbr0-nic -> /usr/share/munin/plugins/if_err_
lrwxrwxrwx. 1 root root 28 25. Jul 12:43 if_netboot -> /usr/share/munin/plugins/if_
lrwxrwxrwx. 1 root root 28 25. Jul 12:43 if_virbr0 -> /usr/share/munin/plugins/if_
lrwxrwxrwx. 1 root root 28 25. Jul 12:43 if_virbr0-nic -> /usr/share/munin/plugins/if_
lrwxrwxrwx. 1 root root 35 25. Jul 12:43 interrupts -> /usr/share/munin/plugins/interrupts
lrwxrwxrwx. 1 root root 33 25. Jul 12:43 irqstats -> /usr/share/munin/plugins/irqstats
lrwxrwxrwx. 1 root root 29 25. Jul 12:43 load -> /usr/share/munin/plugins/load
lrwxrwxrwx. 1 root root 31 25. Jul 12:43 lpstat -> /usr/share/munin/plugins/lpstat
lrwxrwxrwx. 1 root root 31 25. Jul 12:43 memory -> /usr/share/munin/plugins/memory
lrwxrwxrwx. 1 root root 32 25. Jul 12:43 netstat -> /usr/share/munin/plugins/netstat
lrwxrwxrwx. 1 root root 36 25. Jul 12:43 nfs4_client -> /usr/share/munin/plugins/nfs4_client
lrwxrwxrwx. 1 root root 35 25. Jul 12:43 nfs_client -> /usr/share/munin/plugins/nfs_client
lrwxrwxrwx. 1 root root 29 25. Jul 12:43 nfsd -> /usr/share/munin/plugins/nfsd
lrwxrwxrwx. 1 root root 30 25. Jul 12:43 nfsd4 -> /usr/share/munin/plugins/nfsd4
lrwxrwxrwx. 1 root root 35 25. Jul 12:43 open_files -> /usr/share/munin/plugins/open_files
lrwxrwxrwx. 1 root root 36 25. Jul 12:43 open_inodes -> /usr/share/munin/plugins/open_inodes
lrwxrwxrwx. 1 root root 42 25. Jul 12:43 postfix_mailqueue -> /usr/share/munin/plugins/postfix_mailqueue
lrwxrwxrwx. 1 root root 43 25. Jul 12:43 postfix_mailvolume -> /usr/share/munin/plugins/postfix_mailvolume
lrwxrwxrwx. 1 root root 34 25. Jul 12:43 processes -> /usr/share/munin/plugins/processes
lrwxrwxrwx. 1 root root 33 25. Jul 12:43 proc_pri -> /usr/share/munin/plugins/proc_pri
lrwxrwxrwx. 1 root root 43 25. Jul 12:43 sendmail_mailqueue -> /usr/share/munin/plugins/sendmail_mailqueue
lrwxrwxrwx. 1 root root 43 25. Jul 12:43 sendmail_mailstats -> /usr/share/munin/plugins/sendmail_mailstats
lrwxrwxrwx. 1 root root 45 25. Jul 12:43 sendmail_mailtraffic -> /usr/share/munin/plugins/sendmail_mailtraffic
lrwxrwxrwx. 1 root root 29 25. Jul 12:43 swap -> /usr/share/munin/plugins/swap
lrwxrwxrwx. 1 root root 32 25. Jul 12:43 threads -> /usr/share/munin/plugins/threads
lrwxrwxrwx. 1 root root 31 25. Jul 12:43 uptime -> /usr/share/munin/plugins/uptime
lrwxrwxrwx. 1 root root 30 25. Jul 12:43 users -> /usr/share/munin/plugins/users
lrwxrwxrwx. 1 root root 31 25. Jul 12:43 vmstat -> /usr/share/munin/plugins/vmstat


I did nothing but installing munin with yum. It is not acceptable that just installing a package floats a mail account with error mails every five minutes!

Please solve it in a way that after installation munin is inactive and produces no error mails and doesn't fill the log files or other files if it was not manually activated by the administrator of the machine.

Comment 45 Edgar Hoch 2014-07-25 10:55:56 UTC
Here the current package versions:

munin-node-2.0.21-1.fc19.noarch
munin-common-2.0.21-1.fc19.noarch
munin-2.0.21-1.fc19.noarch

munin-common-2.0.21-1.fc20.noarch
munin-node-2.0.21-1.fc20.noarch
munin-2.0.21-1.fc20.noarch

Comment 46 d. johnson 2014-07-25 12:39:46 UTC
Edgar, Your issue will vanish if you set the hostname in the config file.  See #c11 for example.

Comment 47 Edgar Hoch 2014-07-25 14:39:51 UTC
(In reply to d. johnson from comment #46)
> Edgar, Your issue will vanish if you set the hostname in the config file. 
> See #c11 for example.

No. I told you why I couldn't do that. See comments #22 and #23 and the others. For short:

- Package installation must not require manual intervention to prevent the package from doing something bad, like flooding the mailbox with error messages. It should work with the files provided by the distribution without errors. It is ok if it does nothing if it is not configured and activated manually, but simple package installation must leave the system in a save state.

- Each package must be installable in a enterprise environment. For example, it should work with dns, ldap, nis as nameservices. This requires that it must not be neccessary to enter a hostname to /etc/hosts other than what is done while system installation, e.g. localhost in case of Fedora 19 and 20.

- A package should handle the case that hostname is not manually set in the config file in another way than flooding the system mailbox with error mails.
  In your case, you didn't handle the case at all because perl prints an internal error ("not a reference"...). Your code should check if a value is a reference before using it as reference...  (I had not looked at your code, but the error message supposes this cause.)


Please, can you modify munin packages so that no manual intervention is required after package installation to prevent munin from doing something bad.

Comment 48 Håkon Løvdal 2014-07-29 17:24:36 UTC
Bug reporter: The problem still exists, just installing the rpm triggers it.
Bug assignee: Well don't just install the rpm. Take some additional, (non-obivious) action.

Patient: My arm hurts when I do this.
Doctor: Well don't do that then!

Can you please explain how you think these two dialouges are different?

Comment 49 Andrew Haveland-Robinson 2014-08-26 17:04:19 UTC
I started to pull my hair out over this too after suddenly getting the dreaded error filling up my mailbox a few days ago:
not a reference at /usr/share/perl5/vendor_perl/Munin/Master/Utils.pm line 866.

Nothing above worked, and there was nothing particularly useful in the munin logs.

Let's try running a plugin:
# munin-run --debug load
# Processing plugin configuration from /etc/munin/plugin-conf.d/df
# Processing plugin configuration from /etc/munin/plugin-conf.d/fw_
# Processing plugin configuration from /etc/munin/plugin-conf.d/hddtemp_smartctl
# Processing plugin configuration from /etc/munin/plugin-conf.d/munin-node
# Processing plugin configuration from /etc/munin/plugin-conf.d/postfix
# Processing plugin configuration from /etc/munin/plugin-conf.d/sendmail
# Setting /rgid/ruid/ to /99/99/
# Setting /egid/euid/ to /99 99/99/
# Setting up environment
Segmentation fault

Other plugins also faulted.

/var/log/audit/audit.log had lots of these lines:
type=ANOM_ABEND msg=audit(1409068502.721:235158): auid=4294967295 uid=0 gid=99 ses=4294967295 pid=23138 comm="/usr/sbin/munin" exe="/usr/bin/perl" sig=11

Then tried this:
# rpm -Va perl*
prelink: /usr/lib64/libperl.so.5.18.2: prelinked file was modified
S.?......    /usr/lib64/libperl.so.5.18.2

Then this:
# prelink /usr/lib64/libperl.so.5.18.2
# rpm -Va perl*
prelink: /usr/lib64/libperl.so.5.18.2: prelinked file was modified
S.?......    /usr/lib64/libperl.so.5.18.2

Still wouldn't prelink. So tried all:
# prelink -av -mR

Running kernel was kernel-3.15.9-200.fc20.x86_64
Had installed kernel-3.15.10-200.fc20.x86_64 the day before the errors started, but hadn't booted into it yet.

Rebooted and plugins are working again, but I can't be sure exactly why munin-node was segfaulting.

Comment 50 d. johnson 2015-03-07 04:57:12 UTC
The current version should set the default hostname and avoid this issue.


Note You need to log in before you can comment on or make changes to this bug.