Bug 1615639
| Summary: | SIGABRT in Rsyslog | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | bugzilla | ||||||||||||||||
| Component: | rsyslog | Assignee: | Jiří Vymazal <jvymazal> | ||||||||||||||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | BaseOS QE Security Team <qe-baseos-security> | ||||||||||||||||
| Severity: | unspecified | Docs Contact: | |||||||||||||||||
| Priority: | unspecified | ||||||||||||||||||
| Version: | 7.5 | CC: | bugzilla, dapospis, dkopecek, jvymazal, rmeggins | ||||||||||||||||
| Target Milestone: | rc | ||||||||||||||||||
| Target Release: | --- | ||||||||||||||||||
| Hardware: | Unspecified | ||||||||||||||||||
| OS: | Unspecified | ||||||||||||||||||
| Whiteboard: | |||||||||||||||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||||||||||||||
| Doc Text: | Story Points: | --- | |||||||||||||||||
| Clone Of: | Environment: | ||||||||||||||||||
| Last Closed: | 2018-11-05 09:58:08 UTC | Type: | Bug | ||||||||||||||||
| Regression: | --- | Mount Type: | --- | ||||||||||||||||
| Documentation: | --- | CRM: | |||||||||||||||||
| Verified Versions: | Category: | --- | |||||||||||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||||||||||
| Embargoed: | |||||||||||||||||||
| Attachments: |
|
||||||||||||||||||
Created attachment 1475679 [details]
abrt email report 2
Created attachment 1475680 [details]
abrt email report 3
Created attachment 1475681 [details]
abrt email report 4
Created attachment 1475682 [details]
abrt email report 5
Created attachment 1475683 [details]
abrt email report 6
The 6 attachments are each an [abrt] log with the subject:
[abrt] rsyslog: rsyslogd killed by SIGABRT
From what I have seen, they are similar, but maybe you will find a trend. It appears to trigger on realloc() but this machine has plenty of available memory:
[root@host ~] >> free -m
total used free shared buff/cache available
Mem: 31905 3832 7186 1015 20887 26472
Swap: 8095 1423 6672
Can you attach your rsyslog configuration? You appear to be using omfwd at least as well as other modules, so I'm assuming you have customized the default configuration. $WorkDirectory /var/lib/rsyslog # default location for work (spool) files $ActionQueueType LinkedList # use asynchronous processing $ActionQueueFileName srvrfwd # set file name, also enables disk mode $ActionResumeRetryCount -1 # infinite retries on insert failure $ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down $DefaultNetstreamDriverCAFile /usr/local/ewheelerinc/secmon-ca.crt $DefaultNetstreamDriver gtls # use gtls netstream driver $ActionSendStreamDriverMode 1 # require TLS for the connection $ActionSendStreamDriverAuthMode x509/name *.* @@(o)loghost.example.com:10516 Created attachment 1475916 [details]
This one is a SEGV instead of an ABRT
We have a bit more information on what might be happening: An outbound firewall was enabled that prevented rsyslog from providing remote logging, so /var/lib/rsyslog where we put the spool files had lots of files in it. We have emptied that directory to see if perhaps there was a corrupt file causing a segfault. If that was the case, then it would still be good to fix the code in a way such that it fails gracefully. Hopefully the SEGV stacktrace is useful. (In reply to bugzilla from comment #11) > We have a bit more information on what might be happening: > > An outbound firewall was enabled that prevented rsyslog from providing > remote logging, so /var/lib/rsyslog where we put the spool files had lots of > files in it. We have emptied that directory to see if perhaps there was a > corrupt file causing a segfault. If that was the case, then it would still > be good to fix the code in a way such that it fails gracefully. Hopefully > the SEGV stacktrace is useful. So the removal of the files helped? Correct, we haven't had any issues since deleting the files. You might be able to reproduce the problem by creating a TLS remote log as above and then iptables -I OUTPUT -j DROP (or maybe -j REJECT). Then pipe noise into |logger -t and see if you can get segfaults. At this point I don't think I have any other information that I can provide since the problem hasn't come up again. |
Created attachment 1475678 [details] abrt email report 1 Description of problem: SIGABRT happens but we're not sure how to reproduce the error. Version-Release number of selected component (if applicable): package: rsyslog-8.24.0-16.el7_5.4 kernel: 3.10.0-862.9.1.el7.x86_64 How reproducible: Intermittent. Steps to Reproduce: 1. unknown. 2. 3. Actual results: SIGABRT Expected results: No SIGABRT Additional info: See attached.