Bug 2143761

Summary: Long lines of stdout cause bufio.Scanner: token too long error
Product: Red Hat Enterprise Linux 8 Reporter: Link Dupont <link>
Component: rhcAssignee: Link Dupont <link>
Status: CLOSED ERRATA QA Contact: Pavol Kotvan <pakotvan>
Severity: medium Docs Contact:
Priority: urgent    
Version: 8.6CC: cmarinea, dkuc, fjansen, pakotvan, tbowling
Target Milestone: rcKeywords: Triaged, ZStream
Target Release: 8.8Flags: pm-rhel: mirror+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: rhc-0.2.1-10.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2144926 2146923 2146924 (view as bug list) Environment:
Last Closed: 2023-05-16 08:35:37 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2146923, 2146924    
Deadline: 2022-12-05   

Description Link Dupont 2022-11-17 18:32:23 UTC 
Description of problem:
When a worker prints a long line of text to stdout, the bufio.Scanner used to read lines of text from the worker's stdout pipe errors. This appears to cause rhcd to stop dispatching further messages correctly.

Version-Release number of selected component (if applicable):
0.2.1

How reproducible:


Steps to Reproduce:
1. Start rhcd, letting it start up rhc-worker-playbook
2. Send a playbook that creates a lot of output

Actual results:
rhcd prints the error message:
2022/11/07 12:47:40 /builddir/build/BUILD/rhc/yggdrasil-0.2.1/cmd/yggd/exec.go:62: cannot read from stdout: bufio.Scanner: token too long

Expected results:
No error should be printed

Additional info:

Originally reported in https://issues.redhat.com/browse/ADVISOR-2872
Comment 1 Link Dupont 2022-11-21 19:09:01 UTC 
At the end of a playbook run, rhc-worker-playbook prints to stdout an array of all ansible-runner events collected during the run. For certain playbooks, this array can become quite large, as it contains the stdout of all the tasks executed during the playbook run. In the case of ADVISOR-2872, the final line of output is 315016 characters long. This exceeds the maximum token size when scanning by lines using a bufio.Scanner by a significant amount. The default token size is 64 * 1024 = 65536. Hence the error message.
Comment 2 Link Dupont 2022-11-22 18:35:25 UTC 
Rewriting the goroutines that read output from workers avoids hitting this bufio.ErrTooLong error. I've included a patch in PR #100 that reads output from workers using a fixed-length buffer of 4096 bytes. It calls Read directly on the stdout and stderr file handles instead of using a bufio.Scanner to read variable-length lines.
Comment 15 errata-xmlrpc 2023-05-16 08:35:37 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (rhc bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:2819