Bug 1744856

Summary: truncation detected, (cannot seek backward to begin of last block) - re-start reading from beginning: Inappropriate ioctl for device
Product: Red Hat Enterprise Linux 7 Reporter: Aniket Bhavsar <abhavsar>
Component: rsyslogAssignee: Jiří Vymazal <jvymazal>
Status: CLOSED ERRATA QA Contact: Dalibor Pospíšil <dapospis>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.7CC: dapospis, jvymazal, rmeggins
Target Milestone: rcKeywords: EasyFix, Patch, Regression, Triaged
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: rsyslog-8.24.0-47.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-03-31 19:10:09 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Aniket Bhavsar 2019-08-23 03:41:05 UTC
Description of problem:
Getting 'truncation detected, (cannot seek backward to begin of last block) - re-start reading from beginning: Inappropriate ioctl for device [v8.24.0-41.el7_7 try http://www.rsyslog.com/e/51 ]' when monitoring files with imfile and “reopenOnTruncate” has been set to “on”. 

Version-Release number of selected component (if applicable):
v8.24.0-41.el7_7

The issue does not appear with rsyslog-8.24.0-34.el7.

How reproducible:


Steps to Reproduce:

1) You can test that issue installing the rsyslog-8.24.0-41.el7_7 release and then create a queue like this one:

module(load="imfile" PollingInterval="10") #needs to be done just once

$template genericlog_rfcmsg_net,"%msg:1:$:%\n"

input(type="imfile"
      File="/tmp/testfile"
      Tag="testfile"
      Severity="debug"
      Facility="local0"
      reopenOnTruncate="on"
      freshstartTail="on")

if $programname == 'testfile' and $syslogfacility-text == 'local0' and $syslogseverity-text == 'debug' then @/dev/null;genericlog_rfcmsg_net

*.* stop

2) Then, create a test file around 2G random size, for example:

#base64 /dev/urandom | head -c 1900000000 > /tmp/testfile && ls -lh /tmp/testfile

3) And then restart the rsyslog service:

#systemctl restart rsyslog

4) Once you restarted the service, open a new terminal a follow the log of rsyslog:

#journalctl -u rsyslog -f

5) On the original terminal, start a loop to make the test file grow:

#watch -n 0.1 -d "base64 /dev/urandom | head -c 1000000 >> /tmp/testfile && ls -lh /tmp/testfile"

Let grow the file till around 5.5G and you can see on the journalctl file that the rsyslog will complain with some like this:

Aug 23 08:51:59 rhel7u5-1.gsslab.pnq.redhat.com rsyslogd[1661]: file '/tmp/testfile': truncation detected, (cannot seek backward to begin of last block) - re-start reading from beginning: Inappropriate ioctl for device [v8.24.0-41.el7_7 try http://www.rsyslog.com/e/51 ]

Now, stop the loop, downgrade rsyslog:

yum downgrade rsyslog-8.24.0-34.el7

and repeat the whole test recreating the testfile. You will see that rsyslog will not complain about anything.

Problem:
Re-reading the whole file it's a problem and generate duplicate entry on the destination file, and also is a problem in matter of network if the original file is shared somehow via network.

Actual results:

Journal logs:
file '/tmp/testfile': truncation detected, (cannot seek backward to begin of last block) - re-start reading from beginning: Inappropriate ioctl for device [v8.24.0-41.el7_7 try http://www.rsyslog.com/e/51 ].

When I visited 'http://www.rsyslog.com/e/51' found the description of this:

~~~~~
File has been truncated This is not a real error but rather a warning message. Most probably it occurs when monitoring files with imfile and “reopenOnTruncate” has been set to “on”. In this case, it indicates truncation of the file has been detected, and as such imfile begins to read it’s content from the beginning
~~~~~

Comment 13 errata-xmlrpc 2020-03-31 19:10:09 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:1000