Bug 2044718

Summary: Pod complains vsphere volume vmdk does not exist, but volume is already mounted to the node
Product: OpenShift Container Platform Reporter: Palash Khaire <pkhaire>
Component: StorageAssignee: Hemant Kumar <hekumar>
Storage sub component: Storage QA Contact: Wei Duan <wduan>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: unspecified CC: aos-bugs, hekumar, jsafrane
Version: 4.7   
Target Milestone: ---   
Target Release: 4.10.z   
Hardware: All   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-04-12 08:10:44 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Palash Khaire 2022-01-25 03:49:24 UTC 
Description of problem:  Pod complains vsphere volume vmdk does not exist, but volume is already mounted to the node

How reproducible:

The scene is in case the pod is restarted or is evicted and if it is scheduled on the same node the pod will observe the error "vmdk not found". 
But in case the pod is scheduled or forced to schedule on a node other than the old node, the pod will start properly.
For testing purposes, we saw that the pod was currently running on the node <node-name> and it was rescheduled on the same node after we force deleted it we observed the same "VMDK not found" error. 
Also, we double-checked, we validated that the vmdk existed in the correct datastore as well as the same datastore was also attached to all the ESXi hosts in the cluster.

Additional info: 

1. This specifically happens with Kafka cluster.
2. All the VM nodes all have access to the datastore "Datastore-name" and we can list the volume "xxx-6gpp5-dynamic-pvc-xxx.vmdk" in the kubevols folder.

I will add the relevant details and the necessary data in the private notes.
Comment 10 Wei Duan 2022-04-07 07:26:49 UTC 
Marked as verified according to https://bugzilla.redhat.com/show_bug.cgi?id=2044718#c8
Comment 12 errata-xmlrpc 2022-04-12 08:10:44 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.10.9 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:1241
Comment 13 Red Hat Bugzilla 2023-09-15 01:19:03 UTC 
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 500 days