Bug 1306116

Summary:	rsyslog crashes in tplToString
Product:	Red Hat Enterprise Linux 6	Reporter:	Susant Sahani <ssahani>
Component:	rsyslog7	Assignee:	Tomas Heinrich <theinric>
Status:	CLOSED INSUFFICIENT_DATA	QA Contact:	BaseOS QE Security Team <qe-baseos-security>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	6.7	CC:	btotty, cww, pvrabec, rsroka, ssahani
Target Milestone:	rc
Target Release:	6.8
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-07-20 13:55:51 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1269194, 1343211

Description Susant Sahani 2016-02-10 04:13:21 UTC

Description of problem:

rsyslog crashes in 

(gdb) bt
#0  0x00007fbdbf86a742 in memcpy () from /lib64/libc.so.6
#1  0x00007fbdc0e195cd in strgen (pMsg=<value optimized out>, ppBuf=0x7fbdbd39c020, pLenBuf=0x7fbdbd39c070) at /usr/include/bits/string3.h:52
#2  0x00007fbdc0e529d6 in tplToString (pTpl=0x7fbdc1a5caa0, pMsg=0x7fbdac01a650, ppBuf=0x7fbdbd39c020, pLenBuf=<value optimized out>, ttNow=0x7fbdb8bb2920) at ../template.c:240
#3  0x00007fbdc0e4e687 in prepareDoActionParams (pAction=0x7fbdc1a6bc00, pBatch=<value optimized out>, pbShutdownImmediate=0x0) at ../action.c:822
#4  prepareBatch (pAction=0x7fbdc1a6bc00, pBatch=<value optimized out>, pbShutdownImmediate=0x0) at ../action.c:1231
#5  processBatchMain (pAction=0x7fbdc1a6bc00, pBatch=<value optimized out>, pbShutdownImmediate=0x0) at ../action.c:1282
#6  0x00007fbdc0e4c5cf in doQueueEnqObjDirectBatch (pAction=0x7fbdc1a6bc00, pBatch=0x7fbdc1aa1588) at ../action.c:1660
#7  doSubmitToActionQBatch (pAction=0x7fbdc1a6bc00, pBatch=0x7fbdc1aa1588) at ../action.c:1678

Version-Release number of selected component (if applicable):
cat installed-rpms | grep rsyslog
rsyslog7-7.4.10-3.el6_7.1.x86_64                            Mon Jan 11 17:29:48 2016


Actual results:
rsyslog crashes

Expected results:
rsyslog should not crash

Additional info:
Attached coredump and SOS

Comment 2 Susant Sahani 2016-02-10 04:15:00 UTC

(gdb) bt
#0  0x00007fbdbf86a742 in memcpy () from /lib64/libc.so.6
#1  0x00007fbdc0e195cd in strgen (pMsg=<value optimized out>, ppBuf=0x7fbdbd39c020, pLenBuf=0x7fbdbd39c070) at /usr/include/bits/string3.h:52
#2  0x00007fbdc0e529d6 in tplToString (pTpl=0x7fbdc1a5caa0, pMsg=0x7fbdac01a650, ppBuf=0x7fbdbd39c020, pLenBuf=<value optimized out>, ttNow=0x7fbdb8bb2920) at ../template.c:240
#3  0x00007fbdc0e4e687 in prepareDoActionParams (pAction=0x7fbdc1a6bc00, pBatch=<value optimized out>, pbShutdownImmediate=0x0) at ../action.c:822
#4  prepareBatch (pAction=0x7fbdc1a6bc00, pBatch=<value optimized out>, pbShutdownImmediate=0x0) at ../action.c:1231
#5  processBatchMain (pAction=0x7fbdc1a6bc00, pBatch=<value optimized out>, pbShutdownImmediate=0x0) at ../action.c:1282
#6  0x00007fbdc0e4c5cf in doQueueEnqObjDirectBatch (pAction=0x7fbdc1a6bc00, pBatch=0x7fbdc1aa1588) at ../action.c:1660
#7  doSubmitToActionQBatch (pAction=0x7fbdc1a6bc00, pBatch=0x7fbdc1aa1588) at ../action.c:1678
#8  0x00007fbdc0e4c709 in doSubmitToActionQNotAllMarkBatch (pAction=<value optimized out>, pBatch=0x7fbdc1aa1588) at ../action.c:1598
#9  0x00007fbdc0e46f11 in execCall (root=<value optimized out>, pBatch=0x7fbdc1aa1588, active=<value optimized out>) at ruleset.c:293
#10 scriptExec (root=<value optimized out>, pBatch=0x7fbdc1aa1588, active=<value optimized out>) at ruleset.c:562
#11 0x00007fbdc0e46ec4 in freeActive (root=<value optimized out>, pBatch=0x7fbdc1aa1588, active=<value optimized out>) at ruleset.c:223
#12 execPROPFILT (root=<value optimized out>, pBatch=0x7fbdc1aa1588, active=<value optimized out>) at ruleset.c:521
#13 scriptExec (root=<value optimized out>, pBatch=0x7fbdc1aa1588, active=<value optimized out>) at ruleset.c:571
#14 0x00007fbdc0e474c6 in processBatchMultiRuleset (pBatch=0x7fbdc1aa1588) at ruleset.c:206
#15 processBatch (pBatch=0x7fbdc1aa1588) at ruleset.c:604
#16 0x00007fbdc0e0ed8a in msgConsumer (notNeeded=<value optimized out>, pBatch=0x7fbdc1aa1588, pbShutdownImmediate=<value optimized out>) at syslogd.c:607
#17 0x00007fbdc0e45e6b in ConsumerReg (pThis=0x7fbdc1aa1060, pWti=0x7fbdc1aa1560) at queue.c:1870
#18 0x00007fbdc0e40eb6 in wtiWorker (pThis=0x7fbdc1aa1560) at wti.c:318
#19 0x00007fbdc0e409a2 in wtpWrkrExecCleanup (arg=0x7fbdc1aa1560) at wtp.c:310
#20 wtpWorker (arg=0x7fbdc1aa1560) at wtp.c:390
#21 0x00007fbdc07af9d1 in start_thread () from /lib64/libpthread.so.0
#22 0x00007fbdbf8c98fd in ?? () from /lib64/libc.so.6
#23 0x0000000000000000 in ?? ()

Comment 4 Susant Sahani 2016-02-10 04:25:49 UTC

Created attachment 1122659 [details]
rsyslog conf

Comment 7 Karel Srot 2016-06-03 08:43:47 UTC

Hi Tomas,
could you please review this ticket. The discussion in #c5 is several years old.

Bryan,
were there any updates since Feb?

Comment 8 Tomas Heinrich 2016-06-06 12:47:31 UTC

The configuration references $MaxMessageSize, which could cause a segfault and which was fixed in 7.4.10-4.

Also, this section

> $ActionQueueType LinkedList
> $ActionQueueSize 1000000
> $ActionQueueWorkerThreads 25
> $ActionQueueDequeueBatchSize 5000
> $ActionQueueSaveOnShutdown on
>
> $ActionResumeRetryCount -1

is misplaced.

They should fix their configuration and update to the latest version before pursuing this further.