+++ This bug was initially created as a clone of Bug #2016460 +++ Description of problem: STF 1.3 configured to monitor multiple OSP 16 clouds with out-of-the-box configuration (i.e. by following the official documentation [1]). The container sg-core of the ceil-meter Smart Gateway fails on regularly on incoming messages with the following errors: > $ oc logs -f default-tst-ceil-meter-smartgateway-5698bb44dc-4z4vs > [...] > 2021-10-21 08:45:20 [DEBUG] failed handling message [error: ceilometer.OsloSchema.Request: OsloMessage: readEscapedChar: invalid escape char after \, error found in #10 byte of ...|ephemeral\|..., bigger context ...|us\": 1, \"ram\": 1024, \"disk\": 40, \"ephemeral\|..., handler: ceilometer-metrics[socket]] > 2021-10-21 08:45:20 [DEBUG] failed handling message [error: ceilometer.OsloSchema.Request: OsloMessage: readStringSlowPath: unexpected end of input, error found in #10 byte of ...|"vcpus\": |..., bigger context ...|": \"11\", \"name\": \"std.cpu1ram1\", \"vcpus\": |..., handler: ceilometer-metrics[socket]] > [...] Full log output is attached, with "dumpMessages" enabled in the SG configuration for increased verbosity. Actual results: Not exhaustive, but what has been observed so far: - some metrics (e.g. cpu_ceilometer) are missing for some overcloud compute nodes in Prometheus/Grafana, resulting in some dashboards (e.g. Virtual Machine dashboard) to work partially (incomplete lists of projects and VMs). Expected results: All the metrics/events of all the overcloud compute nodes can be seen in Prometheus/Grafana.
Verified this is working. Depends on changes tracked in RHBZ#2053681.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: Service Telemetry Framework 1.4 (sg-core-container) security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:0585