Bug 2104938 - The kubernetes nmstate pods didn't start after instance creation
Summary: The kubernetes nmstate pods didn't start after instance creation
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.10
Hardware: Unspecified
OS: Unspecified
high
urgent
Target Milestone: ---
: 4.10.z
Assignee: Christoph Stäbler
QA Contact: Aleksandra Malykhin
URL:
Whiteboard:
: 2106840 2107001 (view as bug list)
Depends On: 2106634
Blocks: 2067081 ztpfw 2103586
TreeView+ depends on / blocked
 
Reported: 2022-07-07 14:07 UTC by Aleksandra Malykhin
Modified: 2022-07-27 10:16 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2106634 (view as bug list)
Environment:
Last Closed: 2022-07-20 07:46:12 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift kubernetes-nmstate pull 287 0 None Draft [release-4.10] Bug 2104938: Add node permission to CSV 2022-07-13 07:33:40 UTC
Red Hat Knowledge Base (Solution) 6967809 0 None None None 2022-07-18 01:22:51 UTC
Red Hat Product Errata RHBA-2022:5568 0 None None None 2022-07-20 07:46:30 UTC

Description Aleksandra Malykhin 2022-07-07 14:07:37 UTC
Description of problem:
The kubernates nmstate pods didn't start after instance creation 

Version-Release number of selected component (if applicable):
Cluster version is 4.10.22

How reproducible:


Steps to Reproduce:
1.  install nmstate operator via CLI or UI
2.  check csv is in Succeeded state
[root@sealusa25 ~]# oc get csv
NAME                                              DISPLAY                       VERSION               REPLACES   PHASE
kubernetes-nmstate-operator.4.10.0-202207041436   Kubernetes NMState Operator   4.10.0-202207041436              Succeeded

3.  create nmstate instance and wait all pods start running

Actual results:
the pods are not started
[root@sealusa25 ~]# oc get pods
NAME                                READY   STATUS    RESTARTS   AGE
nmstate-operator-6bff67d4fb-d55hg   1/1     Running   0          9m44s


Expected results:
All required pods are started


Additional info:
[root@sealusa25 ~]# oc logs nmstate-operator-6bff67d4fb-d55hg

E0707 14:08:07.854061       1 reflector.go:138] sigs.k8s.io/controller-runtime/pkg/cache/internal/informers_map.go:250: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:openshift-nmstate:nmstate-operator" cannot list resource "nodes" in API group "" at the cluster scope

Comment 1 Aleksandra Malykhin 2022-07-07 14:14:43 UTC
must gather http://rhos-compute-node-10.lab.eng.rdu2.redhat.com/logs/BZ_2104938.tar.gz

Comment 2 Aleksandra Malykhin 2022-07-10 10:02:54 UTC
the logs in the comment #1 are from sno cluster

the following logs from regular cluster:
Cluster version is 4.10.0-0.nightly-2022-06-08-150219
kubernetes-nmstate-operator.4.10.0-202207041436



ClusterID: 85f891fe-7c47-4be4-a03c-40f4c98791d3
ClusterVersion: Stable at "4.10.0-0.nightly-2022-06-08-150219"
ClusterOperators:
	All healthy and stable

http://rhos-compute-node-10.lab.eng.rdu2.redhat.com/logs/BZ_2104938_2_must-gather.tar.gz

Comment 3 Aleksandra Malykhin 2022-07-10 10:03:46 UTC
The pods started successfully with the previous knmstate build:
 Kubernetes NMState Operator   4.10.0-202206270836

Comment 4 Ben Nemec 2022-07-13 17:03:01 UTC
*** Bug 2106840 has been marked as a duplicate of this bug. ***

Comment 7 Germano Veit Michel 2022-07-18 01:22:52 UTC
*** Bug 2107001 has been marked as a duplicate of this bug. ***

Comment 8 Aleksandra Malykhin 2022-07-18 05:50:54 UTC
Verified  with Kubernetes NMState Operator   4.10.0-202207140916

All pods are started, no "*v1.Node: nodes is forbidden" errors in the log
$ oc get pods
NAME                                    READY   STATUS    RESTARTS   AGE
nmstate-cert-manager-8587fbb95d-wlvlw   1/1     Running   0          3m18s
nmstate-handler-476q8                   1/1     Running   0          3m18s
nmstate-handler-4xngk                   1/1     Running   0          3m18s
nmstate-handler-8xh6q                   1/1     Running   0          3m18s
nmstate-handler-dt667                   1/1     Running   0          3m18s
nmstate-handler-ghqmz                   1/1     Running   0          3m18s
nmstate-operator-5fd746bd5-msv8j        1/1     Running   0          3m41s
nmstate-webhook-8c4fdfcdb-qbdl8         1/1     Running   0          3m18s
nmstate-webhook-8c4fdfcdb-sdr7v         1/1     Running   0          3m18s

Comment 10 errata-xmlrpc 2022-07-20 07:46:12 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.10.23 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:5568

Comment 11 Jose Gato 2022-07-27 10:16:49 UTC
Hi there,

this new version also fix this bug:

https://bugzilla.redhat.com/show_bug.cgi?id=2102682

similar, but needing other perm.


Note You need to log in before you can comment on or make changes to this bug.