Bug 428775 - rsyslog fails to operate after logrotate if nonexistent pathnames are in config
rsyslog fails to operate after logrotate if nonexistent pathnames are in config
Status: CLOSED CURRENTRELEASE
Product: Fedora
Classification: Fedora
Component: rsyslog (Show other bugs)
8
All Linux
low Severity low
: ---
: ---
Assigned To: Peter Vrabec
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2008-01-15 00:10 EST by Oleg Drokin
Modified: 2008-02-15 21:15 EST (History)
1 user (show)

See Also:
Fixed In Version: 2.0.2-1.fc8
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2008-02-15 21:15:05 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
rsyslog.conf always reproducinf the problem. (1.06 KB, text/plain)
2008-01-16 21:08 EST, Oleg Drokin
no flags Details

  None (edit)
Description Oleg Drokin 2008-01-15 00:10:11 EST
Description of problem:

I found today wedged rsyslogd on one of my boxes, /var/log/messages wasfilled
with this:
Jan 13 04:58:56 premordor rsyslogd: [origin software="rsyslogd" swVersion="1.19.
11" x-pid="31844"][x-configInfo udpReception="No" udpPort="514" tcpReception="No
" tcpPort="0"] restart
Jan 13 04:58:56 premordor rsyslogd:select: Bad file descriptor
Jan 13 04:59:27 premordor rsyslogd:last message repeated 3554475 times
Jan 13 05:00:28 premordor rsyslogd:last message repeated 7198337 times
...
filled for 2 days.
I then restarted the rsyslog service and found this in the /var/log/messages
that did not look very healthy too:
Jan 15 06:41:00 premordor rsyslogd:last message repeated 4846290 times
^@^@^@^@^@^@^@^@6:41:00 premordor rsyslogd:last message repeated 4846290 times
Jan 15 06:41:00 premordor rsyslogd: [origin software="rsyslogd"
swVersion="1.19.11" x-pid="31844"] exiting on signal 15.


Version-Release number of selected component (if applicable):
rsyslog-1.19.11-3.fc8

Further experiments shown this is 100% reproducible:
I did killall -HUP rsyslogd
and got same symtoms in the log.

Further investigation shown that I misspelled a filename in config and
essentially had this line in /etc/rsyslog.conf:
local7.*                                                /-var/log/boot.log

Instead of some complaining on nonexistent pathname, I was getting hangs on
sighup which cannot be right, I think.
Comment 1 Tomas Heinrich 2008-01-16 07:05:18 EST
I couldn't reproduce this with your steps and rsyslog version, however, I got the same 
behavior with upstream version 1.21.2, selector like:
*.*    @1.2.3.4
and hupping rsyslog.
The absence of directory "/-var" can't be the reason because rsyslog creates missing 
directories by default (it can be controled using the $CreateDirs directive, but seems 
not to be documented anywhere).
Can you please construct a minimal configuration file with which you can reproduce this 
and specially try the UDP forwarding selector. Also check the version you are using, 
output of rsyslogd -v would be helpful.
Comment 2 Oleg Drokin 2008-01-16 21:08:13 EST
Created attachment 291935 [details]
rsyslog.conf always reproducinf the problem.

I use rsyslog package that is latest in fedora at the time of bug filing:
[root@premordor ~]# rsyslogd -v
rsyslogd 1.19.11, compiled with:
	FEATURE_PTHREADS (dual-threading):	Yes
	FEATURE_REGEXP: 			Yes
	FEATURE_LARGEFILE:			Yes
	FEATURE_NETZIP (message compression):	Yes
	SYSLOG_INET (Internet/remote support):	Yes
	FEATURE_DEBUG (debug build, slow code): No

See http://www.rsyslog.com for more information.
[root@premordor ~]# rpm -qa | grep rsyslog
rsyslog-1.19.11-3.fc8

See my config file that reproduces problem all the time.
Instructions:
service rsyslog stop
put the file to /etc
service rsyslog start

watch how everything appeared to start normally, no error messages in
/var/log/messages, no /-var hierarchy created.
now killall -HUP rsyslog
watch rsyslogd eating 100% cpu and filling /var/log/messages with stuff.
service rsyslog start
Comment 3 Oleg Drokin 2008-01-16 21:10:18 EST
(In reply to comment #1)
> and specially try the UDP forwarding selector. Also check the version you are
using, 
> output of rsyslogd -v would be helpful.

Why do I need to try udp forwarding selector if I can see problems without it?
Is not the problems I see without it are not enough?
Comment 4 Peter Vrabec 2008-01-17 09:21:01 EST
hi Oleg,

could you try this release:
http://people.redhat.com/pvrabec/rpms/rsyslog/rsyslog-2.0.0-1.fc9.src.rpm

I guess the reason rsyslog couldn't create "/-var" directory was selinux 
policy.
Comment 5 Oleg Drokin 2008-01-18 23:02:01 EST
(In reply to comment #4)
> could you try this release:
> http://people.redhat.com/pvrabec/rpms/rsyslog/rsyslog-2.0.0-1.fc9.src.rpm

I tried it. The hang and tons of messages are no longer present.
I now also get a message in /var/log/messages:

Jan 19 05:53:58 premordor rsyslogd:/-var/log/boot.log: 

This is of course better than nothing, still I guess intention was to print some
sort of error message there?
Comment 6 Fedora Update System 2008-01-22 11:00:19 EST
rsyslog-2.0.0-1.fc8 has been pushed to the Fedora 8 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update rsyslog'
Comment 7 Peter Vrabec 2008-01-22 12:26:30 EST
Hi Oleg, 

could you test this one, please.

http://koji.fedoraproject.org/packages/rsyslog/2.0.0/2.fc8/src/rsyslog-2.0.0-2.fc8.src.rpm
Comment 8 Fedora Update System 2008-01-24 17:01:04 EST
rsyslog-2.0.0-2.fc8 has been pushed to the Fedora 8 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update rsyslog'
Comment 9 Oleg Drokin 2008-02-01 17:37:00 EST
(In reply to comment #7)
> Hi Oleg, 
> 
> could you test this one, please.
> 
>
http://koji.fedoraproject.org/packages/rsyslog/2.0.0/2.fc8/src/rsyslog-2.0.0-2.fc8.src.rpm

This one seems to be printing error message as expected.
Thanks.
Comment 10 Fedora Update System 2008-02-13 12:40:57 EST
rsyslog-2.0.2-1.fc8 has been submitted as an update for Fedora 8
Comment 11 Fedora Update System 2008-02-15 21:15:00 EST
rsyslog-2.0.2-1.fc8 has been pushed to the Fedora 8 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.