[Description of problem] When using Kibana, it's possible to read the message: "Payload content length greater than maximum allowed" [Version-Release number of selected component (if applicable):] OCP 4.5.7 [How reproducible:] The message error is similar to the redacted in this KCS [1] where it has a workaround for OCP 3.11 modifying the Kibana DC doing: ~~~ $ oc edit dc/logging-kibana ... - env: - name: SERVER_MAXPAYLOADBYTES value: "2097154" ... ~~~ But, in OCP 4.x, it's not possible to modify this value being affected the user by it. [Actual results] The Kibana error "Payload content length greater than maximum allowed" is showed. [Expected results] No error and it's possible to work with Kibana [Additional info] It seems that Kibana has by default a server.maxPayloadBytes defined by default. When this limit is reached, it throws the error showed above. Usually, it's easier to workaround this issue if you can modify the kibana config, but in OCP 4.x, it's not possible since the operator won't allow this change in a "Managed" status. [1] https://access.redhat.com/solutions/4848311
The shortterm work around to this issue is to go unmanaged and add the needed env var to the Kibana deployment
Hello Jeff, I was testing it on a cluster like this: 1. Moved the CLO to Unmanaged ~~~ $ oc edit clusterlogging instance ... spec (...) managementState: Unmanaged ~~~ 2. Moved Kibana to Unmanaged ~~~ $ oc edit kibana Kibana ... spec (...) managementState: Unmanaged ~~~ 3. Edited kibana deployment: ~~~ $ oc edit deployment.apps/kibana (...) - env: - name: SERVER_MAXPAYLOADBYTES value: "2097154" ~~~ 4. Checking if this value was taken by Kibana, it appears this value in the configuration file, but, it's not taking efect since the line is starting by '#' ~~~ $ oc rsh <kibana pod> $ grep -i maxpayload config/* config/kibana.yml:#server.maxPayloadBytes: 2097154 config/kibana_new.yml:#server.maxPayloadBytes: 1048576 ~~~ Then, it seems that it's not possible to configure it in Unmanaged status since a different issue is happening with the replacement of the variable here [1] where the code is replacing the value of the variable, but not deleting the '#' symbol that it's by default in the line in the kibana.yaml sed -i "s,${kibana_var}:.*,${kibana_var}:\ ${value}," "${conf_file}" Do you want to open a new bug for this second issue or would it managed in this one? [1] https://github.com/openshift/origin-aggregated-logging/blob/570778764d3b64a1ca73f8a3b2bd67ff0a65688a/kibana/utils#L75
Verified on elasticsearch-operator.5.0.0-56. The SERVER_MAXPAYLOADBYTES can be changed as Env.
Hello, The PR in this Bugzilla is for allowing to move to Unmanaged and being able to modify the SERVER_MAXPAYLOADBYTES value. This is great, but should a different Bugzilla be created for the definitive fix where it's not needed to move to "Unmanaged"? Best regards, Oscar
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Errata Advisory for Openshift Logging 5.0.0), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:0652
Version 5.2.2-21 still happen The workaround mentioned doesn't work OCP 4.7.15 @ocasalsa