Red Hat Bugzilla – Bug 803550
rsyslog fails to initialize gtls driver because of too many open files
Last modified: 2012-06-20 02:58:54 EDT
Description of problem: when logging to a central log server while using gtls rsyslog fails to initialize after a while because it cannot open any more files Version-Release number of selected component (if applicable): 4.6.2-12.el6.x86_64 How reproducible: always Steps to Reproduce: 1. enable gtls on a client 2. log to a central server 3. wait for logs to happen Actual results: logs are sent to a central server Expected results: after a while, logs stop being sent Additional info: this could be because of http://bugzilla.adiscon.com/show_bug.cgi?id=222
Can you please try these rpms? http://theinric.fedorapeople.org/rsyslog-5.8.7-1.el6.x86_64-rpms.2012-03-15.tar.gz
(In reply to comment #2) > Can you please try these rpms? > http://theinric.fedorapeople.org/rsyslog-5.8.7-1.el6.x86_64-rpms.2012-03-15.tar.gz not to be too paranoid, but can you provide the srpm used to build these? thanks
http://theinric.fedorapeople.org/rsyslog-5.8.7-1.el6.src.rpm Just the rsyslog-5.8.7 sources built for rhel 6.2.
First tests I'm having some problems with. I tried upgrading from 4.6.2 directly but rsyslog doesn't seem to start up while in enforcing mode. Audit reports syslogd_t requesting setsched but has no other denials. I'll see if I can narrow down the problem.
(In reply to comment #5) > First tests I'm having some problems with. I tried upgrading from 4.6.2 > directly but rsyslog doesn't seem to start up while in enforcing mode. Audit > reports syslogd_t requesting setsched but has no other denials. I'll see if I > can narrow down the problem. Doing a little bit of searching I found this: https://bugzilla.redhat.com/show_bug.cgi?id=689089 It looks like the setsched SELinux permission was added in Jun 2011 and the sys_nice was added in Jan 2012 Specifically 153cc80f8760a5e05cf1d5245a369e2da485e52a for sys_nice and 47d5c167a87e9e30b42f2f4a0daaaa15d608afa1 for the setsched I'll create a module for them and test again.
I've successfully installed rsyslog-5.8.7 and am in the process of testing it. One thing to note is that if you are spooling messages under rsyslog-4.6.2 and then upgrade to 5.8.7 rsyslog will silently not start up. There cannot be any messages left to be sent to a central server from the previous rsyslog version.
(In reply to comment #7) > One thing to note is that if you are spooling messages under rsyslog-4.6.2 and > then upgrade to 5.8.7 rsyslog will silently not start up. There cannot be any > messages left to be sent to a central server from the previous rsyslog version. That's interesting. According to the changelog the format of the spool files shouldn't have changed since 4.1.0, but there seems to be a change in this commit: http://git.adiscon.com/?p=rsyslog.git;a=commit;h=4a5a3196fbe4e5a4e9f8dea49f916462adbf3098 I also don't see any mention of this change in the documentation: http://www.rsyslog.com/doc/v5compatibility.html In my tests the daemon simply segfaults, so this looks like a bug. I've also noticed another minor bug related to queues. I'll try to craft a patch, notify upstream and hopefully this will be resolved soon. Thanks for your testing.
Have not seen the too many open files bug since upgrading to 5.8.7 but it's only been a week. Will continue to monitor for this error to come back up.
I've opened separate bugs for the additional issues: bug 813079, bug 813084. I guess the original issue can be considered fixed with the rebase to rsyslog 5.8.10.
After this length I agree that it has been fixed by a rebase.
(In reply to comment #10) > I've opened separate bugs for the additional issues: bug 813079, bug 813084. > I guess the original issue can be considered fixed with the rebase to rsyslog > 5.8.10. What about the rsyslog needing new SELinux permissions? #============= syslogd_t ============== allow syslogd_t self:capability sys_nice; allow syslogd_t self:process setsched; Should that require a new bug?
(In reply to comment #12) > What about the rsyslog needing new SELinux permissions? That should be fixed by now. > Should that require a new bug? Probably not necessary.
Technical note added. If any revisions are required, please edit the "Technical Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. New Contents: Due to an issue in gnutls library, the gtls module leaked a file descriptor every time it was loaded. After reaching the fd limit, no new files or network connections could be opened. The gtls module was marked not to be unloaded during the lifetime of the process. No file descriptors are being leaked.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHSA-2012-0796.html