Bug 2244353

Summary:	[OCP Tracker] [RDR][CEPHFS] volsync-rsync-src pod's are stuck in ContainerCreating with msg for volume is not a mountpoint
Product:	[Red Hat Storage] Red Hat OpenShift Data Foundation	Reporter:	Pratik Surve <prsurve>
Component:	distribution	Assignee:	Madhu Rajanna <mrajanna>
Status:	CLOSED ERRATA	QA Contact:	Aman Agrawal <amagrawa>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	4.14	CC:	amagrawa, ebenahar, edonnell, kbg, mashetty, mrajanna, muagarwa, ndevos, odf-bz-bot, sheggodu
Target Milestone:	---	Keywords:	Tracking
Target Release:	ODF 4.16.0	Flags:	kbg: needinfo-
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	4.16.0-86	Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2024-07-17 13:10:03 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2244409

Description Pratik Surve 2023-10-16 06:43:00 UTC

Description of problem (please be detailed as possible and provide log
snippests):

[RDR][CEPHFS] volsync-rsync-src pod's are stuck in ContainerCreating with msg for volume is not a mountpoint

Events:
  Type     Reason       Age                      From     Message
  ----     ------       ----                     ----     -------
  Warning  FailedMount  7m25s (x1537 over 2d4h)  kubelet  Unable to attach or mount volumes: unmounted volumes=[data], unattached volumes=[], failed to process volumes=[]: timed out waiting for the condition
  Warning  FailedMount  107s (x1566 over 2d4h)   kubelet  MountVolume.SetUp failed for volume "pvc-5cadfa46-6038-4a36-bf6e-aebd3a133911" : rpc error: code = Internal desc = staging path /var/lib/kubelet/plugins/kubernetes.io/csi/openshift-storage.cephfs.csi.ceph.com/91a3d7827df8a706577f16aa678e4fdd181b9ce40d8d9436c21cf5ef6956588b/globalmount for volume 0001-0011-openshift-storage-0000000000000001-03559d4c-9763-49cd-83f6-90d6dae75f1e is not a mountpoint



Version of all relevant components (if applicable):

OCP version:- 4.14.0-0.nightly-2023-10-10-084534
ODF version:- 4.14.0-150
CEPH version:- ceph version 17.2.6-146.el9cp (1d01c2b30b5fd39787bb8804707c4b2e52e30137) quincy (stable)
ACM version:- 2.9.0-185
SUBMARINER version:- devel
VOLSYNC version:- volsync-product.v0.8.0
VOLSYNC method:- destinationCopyMethod: Direct


Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?
yes

Is there any workaround available to the best of your knowledge?


Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?


Can this issue reproducible?


Can this issue reproduce from the UI?


If this is a regression, please provide more details to justify this:


Steps to Reproduce:
1.Deploy RDR cluster
2.Run cephfs workload
3.check pod status in 1-2 days


Actual results:

volsync-rsync-tls-src-busybox-pvc-20-lbw4w   0/1     ContainerStatusUnknown   1          2d5h
volsync-rsync-tls-src-busybox-pvc-20-z6hp7   0/1     ContainerCreating        0          2d5h


Events:
  Type     Reason       Age                      From     Message
  ----     ------       ----                     ----     -------
  Warning  FailedMount  7m25s (x1537 over 2d4h)  kubelet  Unable to attach or mount volumes: unmounted volumes=[data], unattached volumes=[], failed to process volumes=[]: timed out waiting for the condition
  Warning  FailedMount  107s (x1566 over 2d4h)   kubelet  MountVolume.SetUp failed for volume "pvc-5cadfa46-6038-4a36-bf6e-aebd3a133911" : rpc error: code = Internal desc = staging path /var/lib/kubelet/plugins/kubernetes.io/csi/openshift-storage.cephfs.csi.ceph.com/91a3d7827df8a706577f16aa678e4fdd181b9ce40d8d9436c21cf5ef6956588b/globalmount for volume 0001-0011-openshift-storage-0000000000000001-03559d4c-9763-49cd-83f6-90d6dae75f1e is not a mountpoint

Expected results:
Pod should not be stuck in ContainerCreating for this long time

Additional info:

Comment 8 Madhu Rajanna 2023-10-31 12:24:46 UTC

*** Bug 2240161 has been marked as a duplicate of this bug. ***

Comment 19 errata-xmlrpc 2024-07-17 13:10:03 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Red Hat OpenShift Data Foundation 4.16.0 security, enhancement & bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2024:4591