Description of problem: having a special check script (perl, can't be unfortunately disclosed) will crash nagios after < 40 min and some executions. Version-Release number of selected component (if applicable): nagios-4.3.2-8.el7.x86_64 How reproducible: Always Steps to Reproduce: 1. configure a special service with a special check script (which acts as a http client) 2. for the configured host, the result from webserver is big (> 500 kByte) and processed in perl (JSON decode/encode + check( Actual results: crash after some executions (not on the first one!) last log lines before the crash: [1502850410] Warning: The check of service '***' on host '***' looks like it was orphaned (results never came back; last_check=1502433038; next_check=1502849690). I'm scheduling an immediate check of the service... [1502850410] wproc: iocache_read() from Core Worker 1337 returned -1: Connection reset by peer [1502850410] wproc: Socket to worker Core Worker 1337 broken, removing [1502850410] Caught SIGSEGV, shutting down... [1502850410] Caught SIGTERM, shutting down... Expected results: not crashing Additional info: Backtrace from coredump: (gdb) bt #0 0x00007fc49f6d223d in fo_reassign_wproc_job () #1 0x00007fc49f730f12 in fanout_destroy () #2 0x00007fc49f6d2e26 in handle_worker_result () #3 0x00007fc49f72fde3 in iobroker_poll () #4 0x00007fc49f6eb651 in event_execution_loop () #5 0x00007fc49f6cbad6 in main () (unfortunately there is no debuginfo provided somehow) Have found in archives similar issue: http://tracker.nagios.org/view.php?id=504 which disappeared there, but not here. Particular check disabled, nagios is not crashing anymore.
I think this bug needs to be opened upstream as the developers there can ask better detailed questions on what your undisclosable script has in it. I have opened it as https://github.com/NagiosEnterprises/nagioscore/issues/432
Peter, I have some questions on github about recompiling and re-checking. This may be too much to ask - so I have another idea. I also have a question in regards to the specific amount of output that the plugin is returning. My suggestion is to change the plugin so that the output (stdout and stderr combined) never reaches more than 524,287 characters. If it already is returning less than that (although that is slightly more than 500 k) then please ignore my request :)
regarding the wproc issue: yes, plugin had by accident created more than 500k output, but this should not result in crashes. Plugin enhanced with output limiter, now it behaves better.
Should I keep this ticket open or close as "fixed by user?"
Good question, in general it should be tested and fixed by upstream as it is not "fixed by user" but simply a workaround was applied.
Ah ok. I wasn't sure if when you fixed it that you did anything in the nagios code. Since you didn't it looks like it will need to addressed upstream. I will keep this open.
Can you supply me with a perl script that can reproduce the issue? I've tried the following: #!/usr/bin/perl print '-' x (800 * 1024); print "\n"; As I assumed this would duplicate the issue well enough - it does not - and I am unable to reproduce. I am running 4.3.2 compiled.
nagios-4.4.2-3.el7 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2018-0346a55d0f
nagios-4.4.2-3.fc29 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2018-42555731d2
nagios-4.4.2-3.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-70fe6a4d75
nagios-4.4.2-3.el6 has been submitted as an update to Fedora EPEL 6. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2018-61fe7c6e70
nagios-4.4.2-3.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-70fe6a4d75
nagios-4.4.2-3.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2018-0346a55d0f
nagios-4.4.2-3.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2018-61fe7c6e70
nagios-4.4.2-3.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-42555731d2
nagios-4.4.3-1.el7 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-d661b588d2
nagios-4.4.3-1.el6 has been submitted as an update to Fedora EPEL 6. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-17b388679b
nagios-4.4.3-1.fc29 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2019-376ecc221c
nagios-4.4.3-1.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2019-0b44528ff1
nagios-4.4.3-1.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-d661b588d2
nagios-4.4.3-1.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-17b388679b
nagios-4.4.3-1.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-0b44528ff1
nagios-4.4.3-1.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-376ecc221c
nagios-4.4.3-1.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.
nagios-4.4.3-1.fc29 has been pushed to the Fedora 29 stable repository. If problems still persist, please make note of it in this bug report.
nagios-4.4.3-1.el6 has been pushed to the Fedora EPEL 6 stable repository. If problems still persist, please make note of it in this bug report.
nagios-4.4.3-1.el7 has been pushed to the Fedora EPEL 7 stable repository. If problems still persist, please make note of it in this bug report.