Bug 1993120 - [release-4.6] Volumes are accidentally deleted along with the machine [vsphere]
Summary: [release-4.6] Volumes are accidentally deleted along with the machine [vsphere]
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Cloud Compute
Version: 4.9
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.6.z
Assignee: Mike Fedosin
QA Contact: Milind Yadav
URL:
Whiteboard:
Depends On: 1993118
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-08-12 12:23 UTC by OpenShift BugZilla Robot
Modified: 2021-11-03 21:01 UTC (History)
0 users

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-11-03 21:01:40 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift machine-api-operator pull 905 0 None None None 2021-08-16 10:40:58 UTC
Red Hat Product Errata RHBA-2021:4009 0 None None None 2021-11-03 21:01:46 UTC

Description OpenShift BugZilla Robot 2021-08-12 12:23:23 UTC
+++ This bug was initially created as a clone of Bug #1990432 +++

Description of problem:

Pod deletion and volume detach happen asynchronously, so pod could be deleted before volume detached from the node.

When deleting a machine, this could cause issues for vsphere-volume, because if the node deleted before volume detaching success, then the underline volume will be deleted together with the Machine.

Expected results: 

After machine deletion its volumes should remain untouched.

Related to https://bugzilla.redhat.com/show_bug.cgi?id=1883993

Upstream issue: https://github.com/kubernetes-sigs/cluster-api/issues/4707

Comment 5 Milind Yadav 2021-10-21 11:47:51 UTC
[miyadav@miyadav vsphere]$ oc project openshift-machine-api
Now using project "openshift-machine-api" on server "https://api.miyadav2110.qe.devcluster.openshift.com:6443".

[miyadav@miyadav vsphere]$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.6.0-0.nightly-2021-10-20-163353   True        False         8m14s   Cluster version is 4.6.0-0.nightly-2021-10-20-163353

[miyadav@miyadav vsphere]$ oc create -f pvc.yaml 
persistentvolumeclaim/pvc4 created
[miyadav@miyadav vsphere]$ oc create -f deploymentyaml.yaml 
deployment.apps/dep1 created

[miyadav@miyadav vsphere]$ oc get pods -o wide
NAME                                          READY   STATUS    RESTARTS   AGE   IP            NODE                             NOMINATED NODE   READINESS GATES
cluster-autoscaler-operator-d9546b555-fbml2   2/2     Running   1          45m   10.129.0.5    miyadav2110-bqj7k-master-0       <none>           <none>
dep1-64495756b4-lx9vv                         1/1     Running   0          8s    10.128.2.25   miyadav2110-bqj7k-worker-2mng7   <none>           <none>
machine-api-controllers-78ffbf8794-6pqsp      7/7     Running   4          42m   10.130.0.6    miyadav2110-bqj7k-master-2       <none>           <none>
machine-api-operator-794b4c65c5-jpbqn         2/2     Running   1          45m   10.129.0.2    miyadav2110-bqj7k-master-0       <none>           <none>

[miyadav@miyadav vsphere]$ oc delete machine miyadav2110-bqj7k-worker-2mng7
machine.machine.openshift.io "miyadav2110-bqj7k-worker-2mng7" deleted


[miyadav@miyadav vsphere]$ oc get machines
NAME                             PHASE      TYPE   REGION   ZONE   AGE
miyadav2110-bqj7k-master-0       Running                           49m
miyadav2110-bqj7k-master-1       Running                           49m
miyadav2110-bqj7k-master-2       Running                           49m
miyadav2110-bqj7k-worker-2mng7   Deleting                          45m
miyadav2110-bqj7k-worker-4s9hl                                     25s
miyadav2110-bqj7k-worker-qd2z5   Running                           45m


[miyadav@miyadav vsphere]$ govc datastore.ls -l 'xxxxxxxxxxxxxxxx' | grep 'miyadav2110'
12.0MB    Thu Oct 21 11:36:27 2021  miyadav2110-bqj7k-dynamic-pvc-678a815c-6771-416e-86d1-37d1610db186.vmdk

[miyadav@miyadav vsphere]$ oc get machines
NAME                             PHASE         TYPE   REGION   ZONE   AGE
miyadav2110-bqj7k-master-0       Running                              52m
miyadav2110-bqj7k-master-1       Running                              52m
miyadav2110-bqj7k-master-2       Running                              52m
miyadav2110-bqj7k-worker-4s9hl   Provisioned                          3m4s
miyadav2110-bqj7k-worker-qd2z5   Running                              48m

[miyadav@miyadav vsphere]$ oc get pods -o wide
NAME                                          READY   STATUS    RESTARTS   AGE     IP            NODE                             NOMINATED NODE   READINESS GATES
cluster-autoscaler-operator-d9546b555-fbml2   2/2     Running   1          55m     10.129.0.5    miyadav2110-bqj7k-master-0       <none>           <none>
dep1-64495756b4-mll4f                         1/1     Running   0          6m15s   10.131.0.20   miyadav2110-bqj7k-worker-qd2z5   <none>           <none>
machine-api-controllers-78ffbf8794-6pqsp      7/7     Running   4          51m     10.130.0.6    miyadav2110-bqj7k-master-2       <none>           <none>
machine-api-operator-794b4c65c5-jpbqn         2/2     Running   1          55m     10.129.0.2    miyadav2110-bqj7k-master-0       <none>           <none>
[miyadav@miyadav vsphere]$ oc get machines
NAME                             PHASE     TYPE   REGION   ZONE   AGE
miyadav2110-bqj7k-master-0       Running                          55m
miyadav2110-bqj7k-master-1       Running                          55m
miyadav2110-bqj7k-master-2       Running                          55m
miyadav2110-bqj7k-worker-4s9hl   Running                          6m36s
miyadav2110-bqj7k-worker-qd2z5   Running                          52m
[miyadav@miyadav vsphere]$ 


Moved to PASSED

Comment 8 errata-xmlrpc 2021-11-03 21:01:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6.49 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:4009


Note You need to log in before you can comment on or make changes to this bug.