Bug 2160301

Summary:	[GSS]During OCP upgrade, three OSD pod stuck in CLBO state
Product:	[Red Hat Storage] Red Hat OpenShift Data Foundation	Reporter:	Soumi Mitra <smitra>
Component:	rook	Assignee:	Santosh Pillai <sapillai>
Status:	CLOSED INSUFFICIENT_DATA	QA Contact:
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	4.9	CC:	cgaynor, khover, muagarwa, ocs-bugs, odf-bz-bot, sapillai, tnielsen
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2023-03-14 15:30:25 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Comment 7 Colum Gaynor 2023-01-31 16:46:40 UTC

@Santosh Pillai <sapillai> ----> This issue was associated with one Nokia CloudRAN Support Case where the reported OCP version was ODF 4.10
OCP 4.10 is an important release for several Nokia CloudRAN programmes and they would expect this issue to fixed in ODF 4.10 but also ODF 4.11 and ODF 4.12
There is no Target release Specified for this Bugzilla ?

Nokia need this issue fixed in ODF 4.10 ODF 4.11 and ODF 4.12  
Can you clarify please ODF Engineering Fix Targets

Colum Gaynor - Senior Partner Success Manager, Global Nokia Account

Comment 18 Santosh Pillai 2023-02-08 09:39:22 UTC

I would go one step back to Local Storage Operator (that creates the PVs on these disks). 
When the system reboots, the multipath is not guaranteed to be same. Like, if it was /dev/dm-1 before reboot, it can be /dev/dm-2 after reboot. 
If the device have a stable /dev/disk/by-id/ then LSO should use them instead. Then even if the device name changes on reboot, symlink will keep pointing to the /dev/disk/by-id/. 

Currently I see that customer is using /dev/disk/by-path. 
```
  storageClassDevices:
  - devicePaths:
    - /dev/disk/by-path/pci-0000:81:00.0-sas-phy2-lun-0
    - /dev/disk/by-path/pci-0000:81:00.0-sas-phy3-lun-0
    storageClassName: localblock
    volumeMode: Block
```
So I would just to use the `/dev/disk/by-id` in the LocalVolume (or use LocalVolumeSet) and see if that fixes the upgrade issue.

Comment 19 Santosh Pillai 2023-02-14 15:16:12 UTC

any new update on this?

Comment 21 Santosh Pillai 2023-02-21 10:33:54 UTC

*** Bug 2167651 has been marked as a duplicate of this bug. ***

Comment 22 Travis Nielsen 2023-03-14 15:30:25 UTC

Please reopen when there is more info to investigate