Bug 2008532

Summary:	CreateContainerConfigError:: failed to prepare subPath for volumeMount
Product:	OpenShift Container Platform	Reporter:	Himanshu Madhavani <himadhav>
Component:	Storage	Assignee:	Hemant Kumar <hekumar>
Storage sub component:	Storage	QA Contact:	Penghao Wang <pewang>
Status:	CLOSED ERRATA	Docs Contact:
Severity:	urgent
Priority:	urgent	CC:	adhingra, akaris, aos-bugs, braander, cshepher, dhellard, dwalsh, hekumar, jcrumple, jdobson, jmalde, mduasope, mzali, openshift-bugs-escalate, owasserm, pehunt, pkhaire, rhowe, rkant, scuppett, shan, skanniha, tsmetana, wduan, william.caban, wking
Version:	4.7	Flags:	owasserm: needinfo? owasserm: needinfo? owasserm: needinfo? hekumar: needinfo?
Target Milestone:	---
Target Release:	4.10.0
Hardware:	Unspecified
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-03-10 16:13:54 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2024967

Description Himanshu Madhavani 2021-09-28 13:24:18 UTC

Description of problem:

Container creation fails with 

CreateContainerConfigError:: failed to prepare subPath for volumeMount

~~~~
 hyperkube[52689]: I0928 10:31:48.706092   52689 subpath_linux.go:551] Opening path /var/lib/kubelet/pods/bb21d5df-3856-4352-87f2-74a26583c488/volumes/kubernetes.io~csi/pvc-e9261940-a8ae-46a2-b8f4-4f7d70b7f001/mount/pimcore-var-alias-new
Sep 28 10:31:48 esb hyperkube[52689]: I0928 10:31:48.706826   52689 desired_state_of_world_populator.go:528] Found PVC, ClaimName: "prod-homelte"/"new-prod-homelte-pvc"
Sep 28 10:31:48 esbw hyperkube[52689]: I0928 10:31:48.706873   52689 round_trippers.go:422] GET https://api-int.xyz:6443/api/v1/namespaces/prod-homelte/persistentvolumeclaims/new-prod-homelte-pvc
Sep 28 10:31:48 esbwn hyperkube[52689]: I0928 10:31:48.706879   52689 round_trippers.go:429] Request Headers:
Sep 28 10:31:48 esbwnphyperkube[52689]: I0928 10:31:48.706884   52689 round_trippers.go:433]     Accept: application/vnd.kubernetes.protobuf,application/json
Sep 28 10:31:48 esbw hyperkube[52689]: I0928 10:31:48.706889   52689 round_trippers.go:433]     User-Agent: kubelet/v1.20.0+558d959 (linux/amd64) kubernetes/558d959
Sep 28 10:31:48 esbhyperkube[52689]: I0928 10:31:48.707919   52689 mount_linux.go:262] Unmounting /var/lib/kubelet/pods/bb21d5df-3856-4352-87f2-74a26583c488/volume-subpaths/pvc-e9261940-a8ae-46a2-b8f4-4f7d70b7f001/subscription/0
Sep 28 10:31:48 esbd hyperkube[52689]: E0928 10:31:48.709783   52689 kubelet_pods.go:228] failed to prepare subPath for volumeMount "pimcore-var-alias-new" of container "subscription": error ummounting /var/lib/kubelet/pods/bb21d5df-3856-4352-87f2-74a26583c488/volume-subpaths/pvc-e9261940-a8ae-46a2-b8f4-4f7d70b7f001/subscription/0: unmount failed: exit status 32
Sep 28 10:31:48 esb hyperkube[52689]: Unmounting arguments: /var/lib/kubelet/pods/bb21d5df-3856-4352-87f2-74a26583c488/volume-subpaths/pvc-e9261940-a8ae-46a2-b8f4-4f7d70b7f001/subscription/0
Sep 28 10:31:48 esb hyperkube[52689]: Output: umount: /var/lib/kubelet/pods/bb21d5df-3856-4352-87f2-74a26583c488/volume-subpaths/pvc-e9261940-a8ae-46a2-b8f4-4f7d70b7f001/subscription/0: target is busy.
Sep 28 10:31:48 esbw hyperkube[52689]: I0928 10:31:48.709949   52689 event.go:291] "Event occurred" object="prod-homelte/subscription-80-gcl45" kind="Pod" 
~~~~


Version-Release number of selected component (if applicable):
Openshift 4.7

Actual results:

Container creation should succeed.

Expected results:


Additional info:

Additional details in the comments.

Comment 31 Sébastien Han 2021-10-01 15:56:21 UTC

Agreed with Hemant, the permission denied is a different/new issue. Can we get an "oc describe" of one of the ceph-csi plugin resources?
I just want to verify that the ceph-csi resources are using the correct SCC.

Thanks.

Comment 50 Dwayne 2021-11-01 16:50:59 UTC

Also, there is another customer escalation on this BZ 2008532 due to the ACE (Active Customer Escalation, EN-4547). The end-user customer is the Huntington National Bank, and the SFDC case is 03043463.

Comment 62 Peter Hunter 2021-12-22 12:43:17 UTC

I've been added erroneously to this bug by hekumar. I haven't been active in the Redhat community for more than 15 years, and don't even run a Redhat system these days.

Comment 67 errata-xmlrpc 2022-03-10 16:13:54 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0056