Bug 1791988

Summary: After upgrade from 4.3 to 4.4, NTO cannot get resource "nodes" in API group
Product: OpenShift Container Platform Reporter: Simon <skordas>
Component: Node Tuning OperatorAssignee: Jiří Mencák <jmencak>
Status: CLOSED ERRATA QA Contact: Mike Fiedler <mifiedle>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 4.4CC: scuppett, sejug
Target Milestone: ---   
Target Release: 4.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-05-04 11:24:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Simon 2020-01-16 19:59:50 UTC
Description of problem:
After upgrade from 4.3 to 4.4 node tuning operator cannot get resources "node" in API group.

Version-Release number of selected component (if applicable):
4.4

How reproducible:
100%

Steps to Reproduce:
1. Create OCP 4.3 cluster.
2. Update to version 4.4
oc patch clusterversion/version --patch '{"spec":{"upstream":"https://openshift-release.svc.ci.openshift.org/graph"}}' --type=merge

oc adm upgrade --to-image=registry.svc.ci.openshift.org/ocp/release:4.4.0-0.nightly-2020-01-16-103546 --allow-explicit-upgrade --force

3. Check logs in tuned pods.

Actual results:
On each node the same:

I0116 18:16:04.422258    7008 openshift-tuned.go:550] Pod (openshift-kube-storage-version-migrator/migrator-76ff7f7965-6nz6g) labels changed node wide: true
I0116 18:16:05.357354    7008 openshift-tuned.go:408] Dumping labels to /var/lib/tuned/ocp-pod-labels.cfg
I0116 18:16:05.358560    7008 openshift-tuned.go:441] Getting recommended profile...
I0116 18:16:05.451877    7008 openshift-tuned.go:638] Active and recommended profile (openshift-node) match.  Label changes will not trigger profile reload.
E0116 18:16:29.348558    7008 openshift-tuned.go:881] Error getting node ip-10-0-48-132.us-east-2.compute.internal: nodes "ip-10-0-48-132.us-east-2.compute.internal" is forbidden: User "system:serviceaccount:openshift-cluster-node-tuning-operator:tuned" cannot get resource "nodes" in API group "" at the cluster scope
I0116 18:16:29.348576    7008 openshift-tuned.go:883] Increasing resyncPeriod to 102
I0116 18:18:11.348742    7008 openshift-tuned.go:209] Extracting tuned profiles
I0116 18:18:11.350230    7008 openshift-tuned.go:739] Resync period to pull node/pod labels: 102 [s]
E0116 18:18:11.355820    7008 openshift-tuned.go:881] Error getting node ip-10-0-48-132.us-east-2.compute.internal: nodes "ip-10-0-48-132.us-east-2.compute.internal" is forbidden: User "system:serviceaccount:openshift-cluster-node-tuning-operator:tuned" cannot get resource "nodes" in API group "" at the cluster scope
I0116 18:18:11.355836    7008 openshift-tuned.go:883] Increasing resyncPeriod to 204
I0116 18:21:35.356153    7008 openshift-tuned.go:209] Extracting tuned profiles

Comment 1 Jiří Mencák 2020-01-17 14:56:27 UTC
Proposed upstream fix:
https://github.com/openshift/cluster-node-tuning-operator/pull/103

Comment 3 Simon 2020-01-28 19:40:24 UTC
Verification positive!

After upgrade from 4.3.0-0.nightly-2020-01-26-134543 -> 4.4.0-0.nightly-2020-01-24-141203

there is no more problems with getting 'node' resources.

Comment 5 errata-xmlrpc 2020-05-04 11:24:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0581