Bug 1874558 - Long log lines cause OpenShift 4 Console to not show logs
Summary: Long log lines cause OpenShift 4 Console to not show logs
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Management Console
Version: 4.5
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: ---
: 4.6.0
Assignee: Jakub Hadvig
QA Contact: Yadan Pei
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-09-01 15:18 UTC by Christopher Wawak
Modified: 2021-02-10 14:12 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Extremely long log lines is a resource would cause performance issues on resource log pages. Consequence: Resource log page would become slow or unresponsive. Fix: Limit the number of chars per log line that are displayed on the resource log page and provide an alternate means to view full log content. Result: Resource log page performs well even when logs contain extremely long lines.
Clone Of:
Environment:
Last Closed: 2020-10-27 16:36:42 UTC
Target Upstream Version:


Attachments (Terms of Use)
Long Pod logs (624.62 KB, image/png)
2020-09-30 05:44 UTC, Yadan Pei
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Github openshift console pull 6752 0 None closed Bug 1874558: Fix resource log performance issue 2021-02-15 11:14:53 UTC
Red Hat Knowledge Base (Solution) 5363391 0 None None None 2020-09-01 15:19:21 UTC
Red Hat Product Errata RHBA-2020:4196 0 None None None 2020-10-27 16:36:57 UTC

Description Christopher Wawak 2020-09-01 15:18:23 UTC
Description of problem:
When application creates long (300kchar+) log messages to STDOUT, eventually logs stop displaying in admin console.

Version-Release number of selected component (if applicable):
OpenShift 4.5

How reproducible:
Every time

Steps to Reproduce:
1. Run application that sends 300k character long log messages
2. Look at pod logs in admin console
3. Look at pod logs in oc logs cli

Actual results:
- Admin console slows down and eventually stops updating within a minute or two.
- oc logs cli shows up-to-the-minute logs

Expected results:
- Able to review logs in admin console

Additional info:

Here's the script I run to trigger this:
#!/bin/sh
# /app/payload is a file with a single 300kchar line

echo "logger test"

while true
  do date;
  cat /app/payload;
  sleep 10;
  date;
done

Comment 2 Christopher Wawak 2020-09-01 20:03:35 UTC
I have a simple, facile container image I created to reproduce this bug: quay.io/cwawak/logspewtest

Comment 3 Jakub Hadvig 2020-09-02 11:00:49 UTC
Based on https://bugzilla.redhat.com/show_bug.cgi?id=1874151 the CLI should be working as expected.

Comment 4 Jon Jackson 2020-09-11 14:59:19 UTC
Haven't had a chance to look into this. Will address in upcoming sprint.

Comment 6 Yadan Pei 2020-09-30 05:44:34 UTC
Created attachment 1717735 [details]
Long Pod logs

1. Create a pod with image quay.io/cwawak/logspewtest
apiVersion: v1
kind: Pod
metadata:
  name: example
spec:
  containers:
    - name: test
      image: quay.io/cwawak/logspewtest
2. View pod logs on console, we can see warining message:

Some lines have been abridged because they are exceptionally long.
To view unabridged log content, you can either open the raw file in another window or download it.

Click on `open the raw file in another window` will open a new tab and complete pod logs are shown there. 
Click on `download it` will open a download dialog and user can download & save complete pod log


Verified on 4.6.0-0.nightly-2020-09-29-170625

Comment 7 Yadan Pei 2020-09-30 05:47:31 UTC
sorry, changed assigned back

Comment 11 errata-xmlrpc 2020-10-27 16:36:42 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196


Note You need to log in before you can comment on or make changes to this bug.