Bug 2042711
| Summary: | [IBMCloud] Machine Deletion Hook cannot work on IBMCloud | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Huali Liu <huliu> |
| Component: | Cloud Compute | Assignee: | Joel Speed <jspeed> |
| Cloud Compute sub component: | Other Providers | QA Contact: | Huali Liu <huliu> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | medium | ||
| Priority: | unspecified | ||
| Version: | 4.10 | ||
| Target Milestone: | --- | ||
| Target Release: | 4.10.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-03-10 16:40:58 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Verified on 4.10.0-0.nightly-2022-01-21-074618, Machine Deletion Hook works and machine stuck in deleting phase now.
Steps:
1.Create a machineset with Machine Deletion Hook, for example
spec:
lifecycleHooks:
preTerminate:
- name: MigrateImportantApp
owner: etcd
liuhuali@Lius-MacBook-Pro huali-test % oc create -f ms1.yaml
machineset.machine.openshift.io/huliu-045-nvsdk-terminate created
2.Machine created with Machine Deletion Hook
liuhuali@Lius-MacBook-Pro huali-test % oc get machine huliu-045-nvsdk-terminate-9n7wp -o yaml
apiVersion: machine.openshift.io/v1beta1
kind: Machine
metadata:
annotations:
machine.openshift.io/instance-state: running
creationTimestamp: "2022-01-24T04:53:10Z"
finalizers:
- machine.machine.openshift.io
generateName: huliu-045-nvsdk-terminate-
generation: 2
labels:
machine.openshift.io/cluster-api-cluster: huliu-045-nvsdk
machine.openshift.io/cluster-api-machine-role: worker
machine.openshift.io/cluster-api-machine-type: worker
machine.openshift.io/cluster-api-machineset: huliu-045-nvsdk-terminate
machine.openshift.io/instance-type: bx2d-4x16
machine.openshift.io/region: eu-gb
machine.openshift.io/zone: eu-gb-1
name: huliu-045-nvsdk-terminate-9n7wp
namespace: openshift-machine-api
ownerReferences:
- apiVersion: machine.openshift.io/v1beta1
blockOwnerDeletion: true
controller: true
kind: MachineSet
name: huliu-045-nvsdk-terminate
uid: b7f8b192-5cdb-4c37-8fd6-f8f9c1f10275
resourceVersion: "84874"
uid: 07e55608-679d-49c2-a2fd-919e12a23825
spec:
lifecycleHooks:
preTerminate:
- name: MigrateImportantApp
owner: etcd
metadata: {}
providerID: ibmvpc://huliu-045-nvsdk/eu-gb-1/huliu-045-nvsdk-terminate-9n7wp
providerSpec:
value:
apiVersion: ibmcloudproviderconfig.openshift.io/v1beta1
credentialsSecret:
name: ibmcloud-credentials
image: huliu-045-nvsdk-rhcos
kind: IBMCloudMachineProviderSpec
metadata:
creationTimestamp: null
primaryNetworkInterface:
securityGroups:
- huliu-045-nvsdk-sg-cluster-wide
- huliu-045-nvsdk-sg-openshift-net
subnet: huliu-045-nvsdk-subnet-compute-eu-gb-1
profile: bx2d-4x16
region: eu-gb
resourceGroup: huliu-045-nvsdk
userDataSecret:
name: worker-user-data
vpc: huliu-045-nvsdk-vpc
zone: eu-gb-1
status:
addresses:
- address: huliu-045-nvsdk-terminate-9n7wp
type: InternalDNS
- address: 10.242.1.6
type: InternalIP
conditions:
- lastTransitionTime: "2022-01-24T04:53:19Z"
status: "True"
type: Drainable
- lastTransitionTime: "2022-01-24T04:54:45Z"
status: "True"
type: InstanceExists
- lastTransitionTime: "2022-01-24T04:53:19Z"
message: 'Terminate operation currently blocked by: [{Name:MigrateImportantApp
Owner:etcd}]'
reason: HookPresent
severity: Warning
status: "False"
type: Terminable
lastUpdated: "2022-01-24T04:56:25Z"
nodeRef:
kind: Node
name: huliu-045-nvsdk-terminate-9n7wp
uid: 5da810a7-509f-4e9f-87b7-5f80441af87d
phase: Running
providerStatus:
conditions:
- lastProbeTime: "2022-01-24T04:53:39Z"
lastTransitionTime: "2022-01-24T04:53:39Z"
message: Machine successfully created
reason: MachineCreationSucceeded
status: "True"
type: MachineCreated
instanceId: 0787_302172ca-ec0a-4f79-a5dd-d3f0391d9473
instanceState: running
liuhuali@Lius-MacBook-Pro huali-test %
3. delete machine and machine stuck in deleting phase
liuhuali@Lius-MacBook-Pro huali-test % oc delete machine huliu-045-nvsdk-terminate-9n7wp
machine.machine.openshift.io "huliu-045-nvsdk-terminate-9n7wp" deleted
^C
liuhuali@Lius-MacBook-Pro huali-test % oc get machine
NAME PHASE TYPE REGION ZONE AGE
huliu-045-nvsdk-drain-m8rzt Running bx2d-4x16 eu-gb eu-gb-1 11m
huliu-045-nvsdk-invalid-4wv85 Provisioned bx2d-4x16 eu-gb eu-gb-1 2m41s
huliu-045-nvsdk-master-0 Running bx2d-4x16 eu-gb eu-gb-1 3h44m
huliu-045-nvsdk-master-1 Running bx2d-4x16 eu-gb eu-gb-2 3h44m
huliu-045-nvsdk-master-2 Running bx2d-4x16 eu-gb eu-gb-3 3h45m
huliu-045-nvsdk-terminate-9n7wp Deleting bx2d-4x16 eu-gb eu-gb-1 14m
huliu-045-nvsdk-terminate-g45mg Running bx2d-4x16 eu-gb eu-gb-1 7m29s
huliu-045-nvsdk-worker-1-2ws4h Running bx2d-4x16 eu-gb eu-gb-1 3h36m
huliu-045-nvsdk-worker-2-wgf7q Running bx2d-4x16 eu-gb eu-gb-2 3h36m
huliu-045-nvsdk-worker-3-nk74c Running bx2d-4x16 eu-gb eu-gb-3 3h36m
liuhuali@Lius-MacBook-Pro huali-test %
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:0056 |
Description of problem: Machine Deletion Hook cannot work on IBMCloud, lifecycleHooks field will be ignored in machine, and machine can be deleted successfully Version-Release number of selected component (if applicable): 4.10.0-0.nightly-2022-01-17-223655 How reproducible: Always Steps to Reproduce: 1.Create a machineset with Machine Deletion Hook, for example spec: lifecycleHooks: preTerminate: - name: MigrateImportantApp owner: etcd 2.Machine created but without Machine Deletion Hook liuhuali@Lius-MacBook-Pro huali-test % oc create -f ms4.yaml machineset.machine.openshift.io/huliu-033-tnd6f-test2 created liuhuali@Lius-MacBook-Pro huali-test % oc get machine huliu-033-tnd6f-test2-tvfn4 -o yaml apiVersion: machine.openshift.io/v1beta1 kind: Machine metadata: annotations: machine.openshift.io/instance-state: running creationTimestamp: "2022-01-19T09:40:01Z" finalizers: - machine.machine.openshift.io generateName: huliu-033-tnd6f-test2- generation: 3 labels: machine.openshift.io/cluster-api-cluster: huliu-033-tnd6f machine.openshift.io/cluster-api-machine-role: worker machine.openshift.io/cluster-api-machine-type: worker machine.openshift.io/cluster-api-machineset: huliu-033-tnd6f-test2 machine.openshift.io/instance-type: bx2d-4x16 machine.openshift.io/region: eu-gb machine.openshift.io/zone: eu-gb-3 name: huliu-033-tnd6f-test2-tvfn4 namespace: openshift-machine-api ownerReferences: - apiVersion: machine.openshift.io/v1beta1 blockOwnerDeletion: true controller: true kind: MachineSet name: huliu-033-tnd6f-test2 uid: 535c8f21-def7-423a-af42-03972d1c17e6 resourceVersion: "557348" uid: b7e2b967-8889-4bfe-b4cf-bc070cfcf62d spec: metadata: {} providerID: ibmvpc://huliu-033-tnd6f/eu-gb-3/huliu-033-tnd6f-test2-tvfn4 providerSpec: value: apiVersion: ibmcloudproviderconfig.openshift.io/v1beta1 credentialsSecret: name: ibmcloud-credentials image: huliu-033-tnd6f-rhcos kind: IBMCloudMachineProviderSpec metadata: creationTimestamp: null primaryNetworkInterface: securityGroups: - huliu-033-tnd6f-sg-cluster-wide - huliu-033-tnd6f-sg-openshift-net subnet: huliu-033-tnd6f-subnet-compute-eu-gb-3 profile: bx2d-4x16 region: eu-gb resourceGroup: huliu-033-tnd6f userDataSecret: name: worker-user-data vpc: huliu-033-tnd6f-vpc zone: eu-gb-3 status: addresses: - address: huliu-033-tnd6f-test2-tvfn4 type: InternalDNS - address: 10.242.129.25 type: InternalIP conditions: - lastTransitionTime: "2022-01-19T09:41:29Z" status: "True" type: InstanceExists lastUpdated: "2022-01-19T09:43:36Z" nodeRef: kind: Node name: huliu-033-tnd6f-test2-tvfn4 uid: 752bfd5a-c2a4-48d4-a5bb-6e04dfc09cba phase: Running providerStatus: conditions: - lastProbeTime: "2022-01-19T09:40:19Z" lastTransitionTime: "2022-01-19T09:40:19Z" message: Machine successfully created reason: MachineCreationSucceeded status: "True" type: MachineCreated instanceId: 07a7_57cf19d7-d3b6-460d-9999-e920540b1732 instanceState: running liuhuali@Lius-MacBook-Pro huali-test % 3.Machine can be deleted successfully liuhuali@Lius-MacBook-Pro huali-test % oc delete machine huliu-033-tnd6f-test2-tvfn4 machine.machine.openshift.io "huliu-033-tnd6f-test2-tvfn4" deleted liuhuali@Lius-MacBook-Pro huali-test % oc get machine NAME PHASE TYPE REGION ZONE AGE huliu-033-tnd6f-master-0 Running bx2d-4x16 eu-gb eu-gb-1 28h huliu-033-tnd6f-master-1 Running bx2d-4x16 eu-gb eu-gb-2 28h huliu-033-tnd6f-master-2 Running bx2d-4x16 eu-gb eu-gb-3 28h huliu-033-tnd6f-test-brvps Running bx2d-4x16 eu-gb eu-gb-3 4h15m huliu-033-tnd6f-test2-g2k5m Provisioned bx2d-4x16 eu-gb eu-gb-3 100s huliu-033-tnd6f-worker-1-7v4bl Running bx2d-4x16 eu-gb eu-gb-1 26h huliu-033-tnd6f-worker-1-btr7t Deleting bx2d-4x16 eu-gb eu-gb-1 28h huliu-033-tnd6f-worker-2-x68w8 Running bx2d-4x16 eu-gb eu-gb-2 28h huliu-033-tnd6f-worker-3-9dqqp Running bx2d-4x16 eu-gb eu-gb-3 28h liuhuali@Lius-MacBook-Pro huali-test % Actual results: Machine without Machine Deletion Hook, machine deleted successfully Expected results: Machine with Machine Deletion Hook, Machine stuck in deleting phase Additional info: https://issues.redhat.com/browse/OCPCLOUD-1348