Description of problem: Nagios dies with SIGSEGV when it does its internally scheduled log rotation (by default configured to be midnight daily) if the livestatus.o broker module from the check-mk-livestatus package is loaded and at least one host has scheduled downtime. Version-Release number of selected component (if applicable): check-mk-livestatus-1.2.2p2-2.el6.i686 How reproducible: Run nagios with livestatus loaded, schedule a host in downtime, wait for log rotation. Steps to Reproduce: 1. install nagios (yum install nagios nagios-plugins-all) 2. install check-mk-livestatus (yum install check-mk-livestatus) 3. in /etc/nagios/nagios.cf add "broker_module=/usr/lib/check_mk/livestatus.o /var/spool/nagios/cmd/live" 4. in /etc/nagios/nagios.cf change "log_rotation_method=d" to "log_rotation_method=h" (to see the crash on the hour instead of midnight) 5. start nagios (service nagios start) 6. schedule localhost (it comes pre-configured in the nagios RPM) in downtime START=`date +%s`; END=`expr $START + 8640000`; echo "[$START] SCHEDULE_HOST_DOWNTIME;localhost;$START;$END;1;0;8640000;root;test" >> /var/spool/nagios/cmd/nagios.cmd You can see this reflected in the log (/var/log/nagios/nagios.log) and the nagios status file (/var/log/nagios/status.dat) with a "hostdowntime" section. 7. wait for an hour changeover (e.g., 08:59 -> 09:00). Actual results: Nagios process dies with SIGSEGV (logged in /var/log/nagios/nagios.log) E.g., "[1396011600] Caught SIGSEGV, shutting down..." Expected results: Nagios process still running. Additional info: This may be (is probably?) a known but unpatched bug. See: http://lists.mathias-kettner.de/pipermail/checkmk-en/2013-December/011087.html
As you may be aware of we don't package the Open Monitoring Distribution (OMD) at all on the EPEL / Fedora repositories. OMD is a bundle of monitoring softwares (Nagios, Icinga, Check-mk itself, mod_python for distributions that have switched to mod_wsgi already until check-mk gets ported to mod_wsgi for WATO to work as expected (and without mod_python)) and the patch you correctly pointed out is related to a bug in Nagios and not in check-mk itself. I'm therefore changing the product from check-mk to nagios so that it'll be hopefully addressed.
Great, thanks for that. I just hope this doesn't fall into one of those "it's their product" vs "it only happens with their module" blackholes.
More information: http://support.nagios.com/forum/viewtopic.php?f=7&t=26168&p=92551&hilit=livestatus&sid=f071033f0fb075074980de555068d38a#p92551 According to the Nagios support web site: > The issue was that at some point in the 3.x branch the internal nagios downtime > structure changed. > > Livestatus is compiled against the old format, hence the problem. Maybe this really is a livestatus bug which can be patched in the livestatus RPM?
A further comment (http://support.nagios.com/forum/viewtopic.php?f=16&t=24172&start=20 but requires login): > Tue Dec 24, 2013 6:09 am > Down graded mk-livestatus to 1.2.0p3 after trying both (1.2.2p3 and 1.2.4b3) > unsucessfully. Log rotation successful even with downtime configurations.
Mike, can you test if the bug is still there with the latest check-mk release being 1.2.4p1 available on epel-testing?
Crash bug still exists in check-mk-1.2.4p2-1.el6 from epel-testing. Using with nagios-3.5.1-1.el6.
Created attachment 1004252 [details] Patch for downtime.h to work with EPEL nagios. Attached patch to stop coredumping with downtime. I expect nobody at Red Hat cares but another customer might come across this and find it useful. We have been running with this for some time now. Problem with livestatus is that Nagios Core has changed the downtime struct in the course of Nagios 3, and check-mk includes and is compiled against an old version. The check-mk authors don't update it because their Nagios fork OMD uses the old struct. You also need some changes like this to the check-mk.spec: Patch1: 02-Downtime-Struct-Update.patch %prep %setup -q -n check_mk-%{version} tar xf agents.tar.gz tar xf web.tar.gz && rm -f web.tar.gz %patch0 -p0 tar zcf web.tar.gz htdocs plugins gunzip livestatus.tar.gz tar xf livestatus.tar %patch1 -p0 tar rf livestatus.tar nagios/downtime.h gzip livestatus.tar.gz
Mike's patch tested / works for us with the following package combination: nagios / epel check-mk / epel check-mk-livestatus repackaged from epel package with Mike's patch. Thanks Mike! Andrea, respectfully suggest that this would be better treated as a check-mk-livestatus issue to be repackaged when it enters EPEL, to provide compatibility with the packaged nagios distribution - currently they're incompatible. - nagios core have changed to the new struct way of doing things on purpose, I would guess they're unlikely to revert. - check-mk author also have no reason to change. Only a suggestion - it's working for us now here, but it seems that this is a forced manual step for every user who wants to use these packages together... --Benji
Thanks Mike for the patch. I'm currently preparing the package for the 1.2.6 release and will ship the patch you kindly provided.
Wonderful, thank you very much.
check-mk-1.2.6p1-1.fc22 has been submitted as an update for Fedora 22. https://admin.fedoraproject.org/updates/check-mk-1.2.6p1-1.fc22
check-mk-1.2.6p1-1.el6 has been submitted as an update for Fedora EPEL 6. https://admin.fedoraproject.org/updates/check-mk-1.2.6p1-1.el6
check-mk-1.2.6p1-1.fc21 has been submitted as an update for Fedora 21. https://admin.fedoraproject.org/updates/check-mk-1.2.6p1-1.fc21
check-mk-1.2.6p1-1.fc20 has been submitted as an update for Fedora 20. https://admin.fedoraproject.org/updates/check-mk-1.2.6p1-1.fc20
check-mk-1.2.6p1-1.el5 has been submitted as an update for Fedora EPEL 5. https://admin.fedoraproject.org/updates/check-mk-1.2.6p1-1.el5
check-mk-1.2.6p1-1.el7 has been submitted as an update for Fedora EPEL 7. https://admin.fedoraproject.org/updates/check-mk-1.2.6p1-1.el7
(In reply to Andrea Veri from comment #9) > Thanks Mike for the patch. I'm currently preparing the package for the 1.2.6 > release and will ship the patch you kindly provided. Hi Andrea, I tried the package from Comment #16 on RHEL7 (already using check-mk-1.2.4p5-2.el7.x86_64 from EPEL). In contrast to the older package in EPEL, the new one requires mod_python, which isn't available on EL7. I don't see anything in the package that actually requires mod_python to run. I think that the dependency on mod_python should be removed, at least on EL7. Thanks! I look forward to being able to use downtimes again!
check-mk-1.2.6p1-2.el7 has been submitted as an update for Fedora EPEL 7. https://admin.fedoraproject.org/updates/check-mk-1.2.6p1-2.el7
check-mk-1.2.6p1-1.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.
check-mk-1.2.6p1-1.fc21 has been pushed to the Fedora 21 stable repository. If problems still persist, please make note of it in this bug report.
check-mk-1.2.6p1-1.fc20 has been pushed to the Fedora 20 stable repository. If problems still persist, please make note of it in this bug report.
check-mk-1.2.6p1-1.el5 has been pushed to the Fedora EPEL 5 stable repository. If problems still persist, please make note of it in this bug report.
check-mk-1.2.6p1-1.el6 has been pushed to the Fedora EPEL 6 stable repository. If problems still persist, please make note of it in this bug report.
check-mk-1.2.6p1-2.el7 has been pushed to the Fedora EPEL 7 stable repository. If problems still persist, please make note of it in this bug report.
nagios-4.0.8-1.fc21 has been submitted as an update to Fedora 21. https://bodhi.fedoraproject.org/updates/FEDORA-2015-16238
nagios-4.0.8-1.el7 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2015-8155
nagios-4.0.8-1.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2015-16239
nagios-4.0.8-1.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2015-16240
nagios-4.0.8-1.el6 has been submitted as an update to Fedora EPEL 6. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2015-8156
nagios-4.0.8-1.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with $ su -c 'dnf --enablerepo=updates-testing update nagios' You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-16239
nagios-4.0.8-1.fc21 has been pushed to the Fedora 21 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with $ su -c 'dnf --enablerepo=updates-testing update nagios' You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-16238
nagios-4.0.8-1.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with $ su -c 'dnf --enablerepo=updates-testing update nagios' You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-16240
nagios-4.0.8-1.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with $ su -c 'dnf --enablerepo=updates-testing update nagios' You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2015-8156
nagios-4.0.8-1.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report. If you want to test the update, you can install it with $ su -c 'dnf --enablerepo=updates-testing update nagios' You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2015-8155
nagios-4.0.8-1.el7 has been pushed to the Fedora EPEL 7 stable repository. If problems still persist, please make note of it in this bug report.
nagios-4.2.4-3.el6 has been submitted as an update to Fedora EPEL 6. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-270f6f4375
nagios-4.2.4-4.el6 has been submitted as an update to Fedora EPEL 6. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-acd2c2af0d
nagios-4.2.4-4.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-acd2c2af0d
nagios-4.3.2-1.el6 has been submitted as an update to Fedora EPEL 6. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-1f2571d162
nagios-4.3.2-1.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-1f2571d162
nagios-4.3.2-3.el6 has been submitted as an update to Fedora EPEL 6. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-16880697fe
nagios-4.3.2-3.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-16880697fe
nagios-4.3.2-5.el6 has been submitted as an update to Fedora EPEL 6. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-328a23d1ed
nagios-4.3.2-5.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-328a23d1ed
nagios-4.3.2-5.el6 has been pushed to the Fedora EPEL 6 stable repository. If problems still persist, please make note of it in this bug report.