Bug 1954973 - "read /proc/pressure/cpu: operation not supported" in node-exporter logs
Summary: "read /proc/pressure/cpu: operation not supported" in node-exporter logs
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Monitoring
Version: 4.8
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: ---
: 4.8.0
Assignee: Pawel Krupa
QA Contact: Junqi Zhao
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-04-29 07:55 UTC by Junqi Zhao
Modified: 2021-07-27 23:05 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-07-27 23:04:52 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift node_exporter pull 86 0 None closed Bug 1957179: Update to 1.1.2 2021-05-06 08:31:48 UTC
Red Hat Product Errata RHSA-2021:2438 0 None None None 2021-07-27 23:05:16 UTC

Description Junqi Zhao 2021-04-29 07:55:42 UTC
Description of problem:
history err="failed to retrieve pressure stats: psi_stats: unavailable for \"cpu\": read /proc/pressure/cpu: operation not supported" for some node-exporter pods. this error does not block the function, and no suh error now
# oc -n openshift-monitoring get pod -o wide| grep node-exporter
node-exporter-8ncrk                            2/2     Running   0          22h     10.0.155.9     ip-10-0-155-9.us-east-2.compute.internal     <none>           <none>
node-exporter-fr7cw                            2/2     Running   0          22h     10.0.174.123   ip-10-0-174-123.us-east-2.compute.internal   <none>           <none>
node-exporter-hjz9b                            2/2     Running   0          22h     10.0.201.226   ip-10-0-201-226.us-east-2.compute.internal   <none>           <none>
node-exporter-jmf9h                            2/2     Running   0          22h     10.0.213.86    ip-10-0-213-86.us-east-2.compute.internal    <none>           <none>
node-exporter-knst7                            2/2     Running   0          5h21m   10.0.194.82    ip-10-0-194-82.us-east-2.compute.internal    <none>           <none>
node-exporter-nprn9                            2/2     Running   0          22h     10.0.134.148   ip-10-0-134-148.us-east-2.compute.internal   <none>           <none>
node-exporter-th9zh                            2/2     Running   0          22h     10.0.183.50    ip-10-0-183-50.us-east-2.compute.internal    <none>           <none>
node-exporter-thnl4                            2/2     Running   0          22h     10.0.131.73    ip-10-0-131-73.us-east-2.compute.internal    <none>           <none>
node-exporter-ts892                            2/2     Running   0          22h     10.0.175.252   ip-10-0-175-252.us-east-2.compute.internal   <none>           <none>

# for i in $(oc -n openshift-monitoring get pod | grep node-exporter | awk '{print $1}'); do echo $i; oc -n openshift-monitoring logs -c node-exporter $i | grep "failed to retrieve pressure stats: psi_stats" | tail -n 3; echo -e "\n"; done
node-exporter-8ncrk


node-exporter-fr7cw
level=error ts=2021-04-29T02:12:50.511Z caller=collector.go:161 msg="collector failed" name=pressure duration_seconds=2.704e-05 err="failed to retrieve pressure stats: psi_stats: unavailable for \"cpu\": read /proc/pressure/cpu: operation not supported"
level=error ts=2021-04-29T02:13:03.982Z caller=collector.go:161 msg="collector failed" name=pressure duration_seconds=4.2044e-05 err="failed to retrieve pressure stats: psi_stats: unavailable for \"cpu\": read /proc/pressure/cpu: operation not supported"
level=error ts=2021-04-29T02:13:05.526Z caller=collector.go:161 msg="collector failed" name=pressure duration_seconds=4.2232e-05 err="failed to retrieve pressure stats: psi_stats: unavailable for \"cpu\": read /proc/pressure/cpu: operation not supported"


node-exporter-hjz9b
level=error ts=2021-04-29T02:18:46.378Z caller=collector.go:161 msg="collector failed" name=pressure duration_seconds=4.0097e-05 err="failed to retrieve pressure stats: psi_stats: unavailable for \"cpu\": read /proc/pressure/cpu: operation not supported"
level=error ts=2021-04-29T02:18:55.938Z caller=collector.go:161 msg="collector failed" name=pressure duration_seconds=7.0716e-05 err="failed to retrieve pressure stats: psi_stats: unavailable for \"cpu\": read /proc/pressure/cpu: operation not supported"
level=error ts=2021-04-29T02:19:01.573Z caller=collector.go:161 msg="collector failed" name=pressure duration_seconds=5.1525e-05 err="failed to retrieve pressure stats: psi_stats: unavailable for \"cpu\": read /proc/pressure/cpu: operation not supported"


node-exporter-jmf9h


node-exporter-knst7


node-exporter-nprn9
level=error ts=2021-04-29T02:08:06.583Z caller=collector.go:161 msg="collector failed" name=pressure duration_seconds=5.3219e-05 err="failed to retrieve pressure stats: psi_stats: unavailable for \"cpu\": read /proc/pressure/cpu: operation not supported"
level=error ts=2021-04-29T02:08:21.600Z caller=collector.go:161 msg="collector failed" name=pressure duration_seconds=4.4011e-05 err="failed to retrieve pressure stats: psi_stats: unavailable for \"cpu\": read /proc/pressure/cpu: operation not supported"
level=error ts=2021-04-29T02:08:36.606Z caller=collector.go:161 msg="collector failed" name=pressure duration_seconds=3.9326e-05 err="failed to retrieve pressure stats: psi_stats: unavailable for \"cpu\": read /proc/pressure/cpu: operation not supported"


node-exporter-th9zh


node-exporter-thnl4
level=error ts=2021-04-29T02:22:52.832Z caller=collector.go:161 msg="collector failed" name=pressure duration_seconds=3.5172e-05 err="failed to retrieve pressure stats: psi_stats: unavailable for \"cpu\": read /proc/pressure/cpu: operation not supported"
level=error ts=2021-04-29T02:23:07.825Z caller=collector.go:161 msg="collector failed" name=pressure duration_seconds=7.0868e-05 err="failed to retrieve pressure stats: psi_stats: unavailable for \"cpu\": read /proc/pressure/cpu: operation not supported"
level=error ts=2021-04-29T02:23:22.852Z caller=collector.go:161 msg="collector failed" name=pressure duration_seconds=4.6535e-05 err="failed to retrieve pressure stats: psi_stats: unavailable for \"cpu\": read /proc/pressure/cpu: operation not supported"


node-exporter-ts892
level=error ts=2021-04-29T02:31:41.178Z caller=collector.go:161 msg="collector failed" name=pressure duration_seconds=5.2468e-05 err="failed to retrieve pressure stats: psi_stats: unavailable for \"cpu\": read /proc/pressure/cpu: operation not supported"
level=error ts=2021-04-29T02:31:46.010Z caller=collector.go:161 msg="collector failed" name=pressure duration_seconds=4.8342e-05 err="failed to retrieve pressure stats: psi_stats: unavailable for \"cpu\": read /proc/pressure/cpu: operation not supported"
level=error ts=2021-04-29T02:31:56.226Z caller=collector.go:161 msg="collector failed" name=pressure duration_seconds=4.034e-05 err="failed to retrieve pressure stats: psi_stats: unavailable for \"cpu\": read /proc/pressure/cpu: operation not supported"


Version-Release number of selected component (if applicable):
upgrade from 4.6.26 -> 4.7.8 -> 4.8.0-fc.1
node-exporter 1.0.1

How reproducible:
first time to see the error

Steps to Reproduce:
1. upgrade from 4.6.26 -> 4.7.8 -> 4.8.0-fc.1
2.
3.

Actual results:


Expected results:


Additional info:
similar issue: https://github.com/prometheus/node_exporter/issues/1961

Comment 2 Pawel Krupa 2021-05-06 08:31:49 UTC
This is fixed with https://github.com/openshift/node_exporter/pull/86

Comment 4 Junqi Zhao 2021-05-07 01:11:39 UTC
tested with 4.8.0-0.nightly-2021-05-06-162549, node-exporter version is 1.1.2 now

# oc -n openshift-monitoring logs -c node-exporter node-exporter-2dpn6 | head
level=info ts=2021-05-06T23:29:13.263Z caller=node_exporter.go:178 msg="Starting node_exporter" version="(version=1.1.2, branch=rhaos-4.8-rhel-8, revision=d37b7a31752973215d003851f458b049f1ceb7bd)"

Comment 7 errata-xmlrpc 2021-07-27 23:04:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2438


Note You need to log in before you can comment on or make changes to this bug.