Description of problem: Very simply, the logrotate config file for cobblerd contains (among other stuffs): ------------------------------------------------------------------------ postrotate if [ -f /var/lock/subsys/cobblerd ]; then /etc/init.d/cobblerd condrestart > /dev/null fi endscript ------------------------------------------------------------------------ Apparently under some circumstances (and I don't know what those are), that conditional restart will hang around indefinitely, with it's stderr still connected to the stdin of an awk that the logrotate system spawns. The end result is that one instance of logrotate will get hung up and never finish. The next time "run-parts" goes to run it, it sees that the last one hasn't finished, and will refuse to start it again. Without manual intervention, it will continue to refuse to run any more logrotate instances, forever. Again, I don't know what is causing the conditional restart to hang around for this particular customer (the sfdc case is 01546883 if you want to reopen it and try to figure it out, that might be a good plan, too), but I found that if you just change that part of the logrotate config file to: ------------------------------------------------------------------------ postrotate if [ -f /var/lock/subsys/cobblerd ]; then /etc/init.d/cobblerd condrestart > /dev/null 2>&1 fi endscript ------------------------------------------------------------------------ (just redirect stderr as well), it at least keeps it from hanging up the whole logrotate system. Version-Release number of selected component (if applicable): cobbler-2.0.7-67 has the same thing in its logrotate file. How reproducible: Not very. It only happened once in a while for this customer, but at least we found a way to prevent it. Steps to Reproduce: 1. 2. 3. Actual results: Rotating the cobblerd log files hangs up the logrotate system so that NOTHING gets rotated. Expected results: We should be able to rotate the cobblerd log files without completely crippling the entire logrotate system. Additional info:
Note: Checking cobbler upstream I see the same as we us in downstream cobbler ... config/cobblerd_rotate- weekly config/cobblerd_rotate- postrotate config/cobblerd_rotate: if [ -f /var/lock/subsys/cobblerd ]; then config/cobblerd_rotate- /etc/init.d/cobblerd condrestart > /dev/null config/cobblerd_rotate- fi
(In reply to Tomas Lestach from comment #1) > Note: Checking cobbler upstream I see the same as we us in downstream > cobbler ... Yeah, I don't doubt that. Neither do I doubt that there was probably something else wrong in the user's cobblerd configuration that made this happen in the first place. I know nothing about cobblerd, I'm a BaseOS guy. When I got involved all the guy knew was that his logrotate wasn't working. Nonetheless, having this as an added safety net against this phenomenon doesn't seem like a bad thing, does it?
Is there some objection to simply redirecting stderr to /dev/null along with stdout? It fixes a pretty serious issue for this customer, and I don't recall seeing anyone come up with a possible down side to this. What's the hold up?
I'm not able to reproduce this bug but I don't see any problem with adding 2>&1 at the end, newer cobbler is doing it anyway. fixed in cobbler.git(release20): 6a30b629e9bee1736d960cab6eab2f6686942b2e
Verified with cobbler-2.0.7-69. Verified sanity-only: logrotate works when called repeatedly. Also, the fix only contains current cobbler rotate config from upstream cobbler and it helped the customer. Note: Logrotate waits for the condrestart task to complete anyway so I don't know how this could help. When I put sleep into condrestart, the logrotate takes <sleep_length> more time to finish. Also, we are getting rid of the error output which seems regressionish, but after consulting this with devel and considering this is current upstream code, we've decided to keep the fix.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:3445