Bug 2101343

Summary:	topolvm-controller get into CrashLoopBackOff few minutes after install
Product:	OpenShift Container Platform	Reporter:	N Balachandran <nibalach>
Component:	Storage	Assignee:	OpenShift Storage Bugzilla Bot <ocp-storage-bot>
Storage sub component:	Kubernetes External Components	QA Contact:	Wei Duan <wduan>
Status:	CLOSED ERRATA	Docs Contact:
Severity:	high
Priority:	unspecified	CC:	jsafrane
Version:	4.11
Target Milestone:	---
Target Release:	4.12.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2023-01-17 19:50:46 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2102782

Description N Balachandran 2022-06-27 08:49:51 UTC

This bug was initially created as a copy of Bug #2101139

I am copying this bug because: 

The fix is in the external-provisioner sidecar.


Description of problem (please be detailed as possible and provide log
snippests):
When installing lvmcluster the following occurs:
oc get pods
NAME                                               READY   STATUS             RESTARTS      AGE
lvm-operator-controller-manager-666d94dfd6-l2w26   3/3     Running            0             12m
topolvm-controller-7f54b85768-4czjk                4/5     CrashLoopBackOff   2 (19s ago)   4m36s
topolvm-node-bjq9n                                 0/4     Init:0/1           0             4m36s
vg-manager-l4q2v                                   1/1     Running            0             4m36s




Version of all relevant components (if applicable):
odf-operator quay.io/rhceph-dev/ocs-registry:4.11.0-103

Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?
Can't install lvmcluster

Is there any workaround available to the best of your knowledge?
No

Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?
1

Can this issue reproducible?
2/2

Can this issue reproduce from the UI?
NR

If this is a regression, please provide more details to justify this:
Cluster could be installed and now not

Steps to Reproduce:
1. install SNO
2. Create namespace openshift-storage. Create operator group and sub for lvm
3. Install LVM cluster


Actual results:
(381venv) [srozen@srozen-lap 4.11]$ oc get pods
NAME                                               READY   STATUS             RESTARTS      AGE
lvm-operator-controller-manager-666d94dfd6-l2w26   3/3     Running            0             12m
topolvm-controller-7f54b85768-4czjk                4/5     CrashLoopBackOff   2 (19s ago)   4m36s
topolvm-node-bjq9n                                 0/4     Init:0/1           0             4m36s
vg-manager-l4q2v                                   1/1     Running            0             4m36s


Expected results:
All pod should be in running state.





RCA:
This is a bug introduced in the recently released external-provisioner v3.2.0. The OCP 4.11 CSI sidecar images were updated last week after which the LVMO build started to fail.


Upstream Issue: https://github.com/kubernetes-csi/external-provisioner/issues/752

Upstream Fix: https://github.com/kubernetes-csi/external-provisioner/pull/753

Comment 1 N Balachandran 2022-06-27 11:10:25 UTC

The upstream PR and its backport have been merged.

https://github.com/kubernetes-csi/external-provisioner/pull/753
https://github.com/kubernetes-csi/external-provisioner/pull/754

Comment 7 Wei Duan 2022-07-04 06:01:05 UTC

According to https://bugzilla.redhat.com/show_bug.cgi?id=2101343#c5, I think it is enough for the verificarion for the default namespace issue in csi-external-provisioner image, will mark it as VERIFIED. Feel free to re-open or raise another BZ if it doesn't work on LVMO-4.11/4.12

Comment 11 errata-xmlrpc 2023-01-17 19:50:46 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.12.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:7399