RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1323916 - logger unnecessarily splits messages sent via stdin into 1024 byte chunks
Summary: logger unnecessarily splits messages sent via stdin into 1024 byte chunks
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: util-linux
Version: 7.2
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: Karel Zak
QA Contact: Radka Brychtova
URL:
Whiteboard:
Depends On:
Blocks: 1203710 1380361
TreeView+ depends on / blocked
 
Reported: 2016-04-05 04:58 UTC by Ryan Sawhill
Modified: 2021-08-30 12:33 UTC (History)
5 users (show)

Fixed In Version: util-linux-2.23.2-36.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-08-01 21:41:11 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 2220491 0 None None None 2016-04-06 02:54:07 UTC
Red Hat Product Errata RHBA-2017:2186 0 normal SHIPPED_LIVE util-linux bug fix and enhancement update 2017-08-01 18:41:36 UTC

Description Ryan Sawhill 2016-04-05 04:58:09 UTC
Description of problem:

  logger splits stdin into 1 KiB chunks. This is especially odd because logger doesn't seem to do any splitting when log messages are passed via argv.

  Why does this matter?
  logger is regularly used by applications as a cheap way to get logs to a local or remote (r)?syslog server. One of the most common users of this is Apache httpd, where httpd spawns a single logger command when started and uses that to send logs. In this situation, it's not at all uncommon for log entries to be larger than 1k. If you google it, you can numerous examples of httpd users complaining about this limitation. 

 I assume the 1024-byte limit is a hold-over from the syslog rfc (http://www.faqs.org/rfcs/rfc3164.html) but in 2016 this is just comical. rsyslog and journald have no such limit.

Version-Release number of selected component (if applicable):

  util-linux-2.23.2-26.el7

Steps to Reproduce:

  1. Use `logger $(printf 'a%.0s' {1..8192})` and notice the journal and rsyslog end up with a single 8k message.
  2. Use `printf 'b%.0s' {1..1025} | logger` and the journal/rsyslog output make it clear that logger splits stdin into 1k-chunks -- you end up with 2 separate log entries.

Expected results:

  I'd like to see logger treat stdin in the same way it treats argv -- i.e., practically unlimited message length.
 
Additional info:

  FYI rsyslog v7 in RHEL 7 is supposed to have a default $MaxMessageSize setting of 2k as seen here: http://www.rsyslog.com/doc/v7-stable/configuration/global/index.html; however, despite our rsyslog-doc package saying the same thing, we've clearly raised the default far above 2k -- probably in response to some customer request. (I didn't check the source.) All the more reason to fix this issue with logger.

Comment 1 Karel Zak 2016-04-05 08:59:37 UTC
The current upstream version provides "--size <bytes>" command line option to overwrite the default (still 1024). Maybe we can backport the option to RHEL7.

I'm not sure if we want to change the default due to backward compatibility.

Comment 2 Ryan Sawhill 2016-04-05 15:06:43 UTC
If we can backport that --size option that upstream already has, that would be awesome and good enough for me; however, I have to say that "backward compatibility" doesn't really make sense to me given that logger accepts virtually unlimited messages when passed on the cmdline as argument(s). ... Which is why I filed this as a bug and not an RFE -- it's inconsistent.

In any case, thanks so much for the prompt response Karel!

Comment 3 Ryan Sawhill 2016-04-12 04:24:48 UTC
Well it took me a few hours to write a logger clone in python that does virtually everything RHEL7's logger cmd does. (I didn't implement --stderr and it only reads input from stdin.)

I have no love for perl, but I want to see how the performance compares, so I spent a few more (half dozen?) much more painful hours doing the same in perl. *shudder*

Neither of these logger clones suffer from the stdin message-splitting limitation of the stock logger command. Both are linked from:
https://access.redhat.com/solutions/2220491

Hopefully we can get an officially-supported method sooner rather than later.

Comment 4 Ryan Sawhill 2016-04-13 17:51:06 UTC
(In reply to Ryan Sawhill from comment #3)
> Well it took me a few hours to write a logger clone in python that does
> virtually everything RHEL7's logger cmd does. (I didn't implement --stderr
> and it only reads input from stdin.)

I have to correct this for the record: I spent a couple more hours on it and my logger.py now accepts logs via cmdline args as well as -f/--file. Also added features from upstream: --id for custom IDs and -e/--skip-empty for fileinput. Finally, added a new --ppid option.

Comment 7 Florian Crouzat 2016-12-30 10:10:07 UTC
So any chance to get the upstream version of logger which supports the --size option backported into RHEL7 any time or even better, any time soon ?

I can reproduce the issue with rhel7.3 and util-linux-2.23.2-33.el7.x86_64

If not, Ryan, any chance you published your python script somewhere so I can try it ? I definitely need a fix/workaround to this 1024-byte limit  ...

Comment 8 Ryan Sawhill 2016-12-30 17:05:35 UTC
(In reply to Florian Crouzat from comment #7)
...
> If not, Ryan, any chance you published your python script somewhere so I can
> try it ? I definitely need a fix/workaround to this 1024-byte limit  ...

Hi Florian [and anyone else with the same question]. The perl & python scripts are linked from the KCS solution in comment #3 -- direct link:

  https://github.com/ryran/loggerclones

Of course the obvious disclaimer applies: I do not offer these in my capacity as a Red Hat employee; treat them as 3rd-party/unsupported/no-warranty -- any further queries about them should be directed to that github repo. :)

Comment 9 Karel Zak 2017-01-03 10:13:04 UTC
Frankly, if you're fine with script as logger replacement than you can probably use logger binary from Fedora where is no such restriction :-)  (or rebuild it from upstream github)

Anyway, from my point of view the --size backport seems fine, going to suggest it for 7.4.

Comment 10 Florian Crouzat 2017-01-03 10:35:22 UTC
(In reply to Karel Zak from comment #9)
> Frankly, if you're fine with script as logger replacement than you can
> probably use logger binary from Fedora where is no such restriction :-)  (or
> rebuild it from upstream github)
> 
> Anyway, from my point of view the --size backport seems fine, going to
> suggest it for 7.4.

That would be great. I'll stay tuned. Will this BZ updates inform me weither or not it will be shipped in 7.4 ?

(In reply to Ryan Sawhill from comment #8)
> https://github.com/ryran/loggerclones

Thanks, I'll wait for a conclusion on the above topic (7.4) before trying your scripts as I'd rather remain in the stable branch, but I'll have a look at it in case the option is not backported.

Comment 15 errata-xmlrpc 2017-08-01 21:41:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2186


Note You need to log in before you can comment on or make changes to this bug.