Bug 2207918

Summary: [RDR] After performing relocate operation some pod stuck in ContainerCreating with msg applyFSGroup failed for vol
Product: [Red Hat Storage] Red Hat OpenShift Data Foundation Reporter: Pratik Surve <prsurve>
Component: cephAssignee: Ilya Dryomov <idryomov>
ceph sub component: RBD-Mirror QA Contact: Sidhant Agrawal <sagrawal>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: unspecified CC: amagrawa, bniver, ebenahar, idryomov, kramdoss, kseeger, muagarwa, odf-bz-bot, sostapov, srangana
Version: 4.13   
Target Milestone: ---   
Target Release: ODF 4.14.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: 4.14.0-108 Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of:
: 2212511 2216186 (view as bug list) Environment:
Last Closed: 2023-11-08 18:50:22 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2212511, 2214973    
Bug Blocks: 2023189    

Description Pratik Surve 2023-05-17 10:01:18 UTC 
Description of problem (please be detailed as possible and provide log
snippets):

[RDR] Pod stuck in ContainerCreating with msg applyFSGroup failed for vol 0001-0011-openshift-storage-0000000000000001-b53903f4-185e-4ee1-8496-57c285416591: readdirent /var/lib/kubelet/pods/530fd454-3a38-4546-86a9-f3cdef8892a1/volumes/kubernetes.io~csi/pvc-1efbe6b3-9320-40e4-8733-45709e334193/mount/linux-4.14.280_1684209883/tools/perf/pmu-events/arch/x86/bonnell: bad message

Version of all relevant components (if applicable):

OCP version:- 4.13.0-0.nightly-2023-05-11-225357
ODF version:- 4.13.0-199
CEPH version:- ceph version 17.2.6-47.el9cp (6add4f24d1eff88e1db808ecdc16fd5b2db96dd4) quincy (stable)
ACM version:- 2.8.0-169
SUBMARINER version:- v0.15.0
VOLSYNC version:- volsync-product.v0.7.1

Does this issue impact your ability to continue to work with the product
(please explain in detail what is the user impact)?

yes
Is there any workaround available to the best of your knowledge?


Rate from 1 - 5 the complexity of the scenario you performed that caused this
bug (1 - very simple, 5 - very complex)?


Can this issue reproducible?


Can this issue reproduce from the UI?


If this is a regression, please provide more details to justify this:


Steps to Reproduce:
1.Deploy RDR cluster
2.Deploy workload https://github.com/red-hat-storage/ocs-workloads/tree/master/rdr/busybox/app-busybox-2/resources/deployment
3.Perform relocate operation


Actual results:

Events:
  Type     Reason       Age                  From     Message
  ----     ------       ----                 ----     -------
  Warning  FailedMount  31m (x496 over 23h)  kubelet  Unable to attach or mount volumes: unmounted volumes=[mypvc], unattached volumes=[mypvc kube-api-access-2mfb4]: timed out waiting for the condition
  Warning  FailedMount  99s (x689 over 23h)  kubelet  MountVolume.SetUp failed for volume "pvc-1efbe6b3-9320-40e4-8733-45709e334193" : applyFSGroup failed for vol 0001-0011-openshift-storage-0000000000000001-b53903f4-185e-4ee1-8496-57c285416591: readdirent /var/lib/kubelet/pods/530fd454-3a38-4546-86a9-f3cdef8892a1/volumes/kubernetes.io~csi/pvc-1efbe6b3-9320-40e4-8733-45709e334193/mount/linux-4.14.280_1684209883/tools/perf/pmu-events/arch/x86/bonnell: bad message


Expected results:


Additional info:
Comment 46 errata-xmlrpc 2023-11-08 18:50:22 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Red Hat OpenShift Data Foundation 4.14.0 security, enhancement & bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:6832