Bug 2008532

Summary: CreateContainerConfigError:: failed to prepare subPath for volumeMount
Product: OpenShift Container Platform Reporter: Himanshu Madhavani <himadhav>
Component: StorageAssignee: Hemant Kumar <hekumar>
Storage sub component: Storage QA Contact: Penghao Wang <pewang>
Status: CLOSED ERRATA Docs Contact:
Severity: urgent    
Priority: urgent CC: adhingra, akaris, aos-bugs, braander, cshepher, dhellard, dwalsh, hekumar, jcrumple, jdobson, jmalde, mduasope, mzali, openshift-bugs-escalate, owasserm, pehunt, pkhaire, rhowe, rkant, scuppett, shan, skanniha, tsmetana, wduan, william.caban, wking
Version: 4.7Flags: owasserm: needinfo?
owasserm: needinfo?
owasserm: needinfo?
hekumar: needinfo?
Target Milestone: ---   
Target Release: 4.10.0   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-03-10 16:13:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2024967    

Description Himanshu Madhavani 2021-09-28 13:24:18 UTC
Description of problem:

Container creation fails with 

CreateContainerConfigError:: failed to prepare subPath for volumeMount

~~~~
 hyperkube[52689]: I0928 10:31:48.706092   52689 subpath_linux.go:551] Opening path /var/lib/kubelet/pods/bb21d5df-3856-4352-87f2-74a26583c488/volumes/kubernetes.io~csi/pvc-e9261940-a8ae-46a2-b8f4-4f7d70b7f001/mount/pimcore-var-alias-new
Sep 28 10:31:48 esb hyperkube[52689]: I0928 10:31:48.706826   52689 desired_state_of_world_populator.go:528] Found PVC, ClaimName: "prod-homelte"/"new-prod-homelte-pvc"
Sep 28 10:31:48 esbw hyperkube[52689]: I0928 10:31:48.706873   52689 round_trippers.go:422] GET https://api-int.xyz:6443/api/v1/namespaces/prod-homelte/persistentvolumeclaims/new-prod-homelte-pvc
Sep 28 10:31:48 esbwn hyperkube[52689]: I0928 10:31:48.706879   52689 round_trippers.go:429] Request Headers:
Sep 28 10:31:48 esbwnphyperkube[52689]: I0928 10:31:48.706884   52689 round_trippers.go:433]     Accept: application/vnd.kubernetes.protobuf,application/json
Sep 28 10:31:48 esbw hyperkube[52689]: I0928 10:31:48.706889   52689 round_trippers.go:433]     User-Agent: kubelet/v1.20.0+558d959 (linux/amd64) kubernetes/558d959
Sep 28 10:31:48 esbhyperkube[52689]: I0928 10:31:48.707919   52689 mount_linux.go:262] Unmounting /var/lib/kubelet/pods/bb21d5df-3856-4352-87f2-74a26583c488/volume-subpaths/pvc-e9261940-a8ae-46a2-b8f4-4f7d70b7f001/subscription/0
Sep 28 10:31:48 esbd hyperkube[52689]: E0928 10:31:48.709783   52689 kubelet_pods.go:228] failed to prepare subPath for volumeMount "pimcore-var-alias-new" of container "subscription": error ummounting /var/lib/kubelet/pods/bb21d5df-3856-4352-87f2-74a26583c488/volume-subpaths/pvc-e9261940-a8ae-46a2-b8f4-4f7d70b7f001/subscription/0: unmount failed: exit status 32
Sep 28 10:31:48 esb hyperkube[52689]: Unmounting arguments: /var/lib/kubelet/pods/bb21d5df-3856-4352-87f2-74a26583c488/volume-subpaths/pvc-e9261940-a8ae-46a2-b8f4-4f7d70b7f001/subscription/0
Sep 28 10:31:48 esb hyperkube[52689]: Output: umount: /var/lib/kubelet/pods/bb21d5df-3856-4352-87f2-74a26583c488/volume-subpaths/pvc-e9261940-a8ae-46a2-b8f4-4f7d70b7f001/subscription/0: target is busy.
Sep 28 10:31:48 esbw hyperkube[52689]: I0928 10:31:48.709949   52689 event.go:291] "Event occurred" object="prod-homelte/subscription-80-gcl45" kind="Pod" 
~~~~


Version-Release number of selected component (if applicable):
Openshift 4.7

Actual results:

Container creation should succeed.

Expected results:


Additional info:

Additional details in the comments.

Comment 31 S├ębastien Han 2021-10-01 15:56:21 UTC
Agreed with Hemant, the permission denied is a different/new issue. Can we get an "oc describe" of one of the ceph-csi plugin resources?
I just want to verify that the ceph-csi resources are using the correct SCC.

Thanks.

Comment 50 Dwayne 2021-11-01 16:50:59 UTC
Also, there is another customer escalation on this BZ 2008532 due to the ACE (Active Customer Escalation, EN-4547). The end-user customer is the Huntington National Bank, and the SFDC case is 03043463.

Comment 62 Peter Hunter 2021-12-22 12:43:17 UTC
I've been added erroneously to this bug by hekumar. I haven't been active in the Redhat community for more than 15 years, and don't even run a Redhat system these days.

Comment 67 errata-xmlrpc 2022-03-10 16:13:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0056