Bug 2042715
| Summary: | [AliCloud] Machine Deletion Hook cannot work on AliCloud | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Huali Liu <huliu> |
| Component: | Cloud Compute | Assignee: | Joel Speed <jspeed> |
| Cloud Compute sub component: | Other Providers | QA Contact: | Huali Liu <huliu> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | medium | ||
| Priority: | medium | ||
| Version: | 4.10 | ||
| Target Milestone: | --- | ||
| Target Release: | 4.10.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-03-10 16:40:58 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Verified on 4.10.0-0.nightly-2022-01-21-074618, Machine Deletion Hook works and machine stuck in deleting phase now.
Steps:
1.Create a machineset with Machine Deletion Hook, for example
spec:
lifecycleHooks:
preTerminate:
- name: MigrateImportantApp
owner: etcd
liuhuali@Lius-MacBook-Pro huali-test % oc create -f ms1.yaml
machineset.machine.openshift.io/huliu-046-kxqvn-test created
2.Machine created with Machine Deletion Hook
liuhuali@Lius-MacBook-Pro huali-test % oc get machine huliu-046-kxqvn-test-9c6lv -o yaml
apiVersion: machine.openshift.io/v1beta1
kind: Machine
metadata:
annotations:
machine.openshift.io/instance-state: Running
creationTimestamp: "2022-01-24T02:04:00Z"
finalizers:
- machine.machine.openshift.io
generateName: huliu-046-kxqvn-test-
generation: 2
labels:
machine.openshift.io/cluster-api-cluster: huliu-046-kxqvn
machine.openshift.io/cluster-api-machine-role: worker
machine.openshift.io/cluster-api-machine-type: worker
machine.openshift.io/cluster-api-machineset: huliu-046-kxqvn-test
machine.openshift.io/instance-type: ecs.g6.large
machine.openshift.io/region: us-east-1
machine.openshift.io/zone: us-east-1a
name: huliu-046-kxqvn-test-9c6lv
namespace: openshift-machine-api
ownerReferences:
- apiVersion: machine.openshift.io/v1beta1
blockOwnerDeletion: true
controller: true
kind: MachineSet
name: huliu-046-kxqvn-test
uid: 1e62b9e0-63bf-4460-a28a-e9cf2f9b90ec
resourceVersion: "41915"
uid: 1cd0da4f-816a-4ea2-8d42-2b0da9bd896c
spec:
lifecycleHooks:
preTerminate:
- name: MigrateImportantApp
owner: etcd
metadata: {}
providerID: alicloud://us-east-1.i-0xi5mkzdhi83bmvep8ql
providerSpec:
value:
apiVersion: alibabacloudmachineproviderconfig.openshift.io/v1beta1
bandwidth: {}
credentialsSecret:
name: alibabacloud-credentials
dedicatedHostId: ""
imageId: m-0xi5sh1y90lsbo8ayl0a
instanceType: ecs.g6.large
kind: AlibabaCloudMachineProviderConfig
metadata:
creationTimestamp: null
ramRoleName: huliu-046-kxqvn-role-worker
regionId: us-east-1
securityGroups:
- tags:
- Key: kubernetes.io/cluster/huliu-046-kxqvn
Value: owned
- Key: GISV
Value: ocp
- Key: sigs.k8s.io/cloud-provider-alibaba/origin
Value: ocp
- Key: Name
Value: huliu-046-kxqvn-sg-worker
subscription: {}
systemDisk:
category: cloud_essd
size: 120
tags:
- Key: kubernetes.io/cluster/huliu-046-kxqvn
Value: owned
- Key: GISV
Value: ocp
- Key: sigs.k8s.io/cloud-provider-alibaba/origin
Value: ocp
userDataSecret:
name: worker-user-data
vSwitch:
tags:
- Key: kubernetes.io/cluster/huliu-046-kxqvn
Value: owned
- Key: GISV
Value: ocp
- Key: sigs.k8s.io/cloud-provider-alibaba/origin
Value: ocp
- Key: Name
Value: huliu-046-kxqvn-vswitch-us-east-1a
zoneId: us-east-1a
status:
addresses:
- address: 10.0.28.183
type: InternalIP
- address: huliu-046-kxqvn-test-9c6lv
type: InternalDNS
conditions:
- lastTransitionTime: "2022-01-24T02:04:01Z"
status: "True"
type: Drainable
- lastTransitionTime: "2022-01-24T02:04:58Z"
status: "True"
type: InstanceExists
- lastTransitionTime: "2022-01-24T02:04:01Z"
message: 'Terminate operation currently blocked by: [{Name:MigrateImportantApp
Owner:etcd}]'
reason: HookPresent
severity: Warning
status: "False"
type: Terminable
lastUpdated: "2022-01-24T02:08:14Z"
nodeRef:
kind: Node
name: huliu-046-kxqvn-test-9c6lv
uid: 991f86d5-7b7f-4099-b9e2-1218085ddc73
phase: Running
providerStatus:
conditions:
- lastTransitionTime: "2022-01-24T02:04:58Z"
message: Machine successfully created
reason: MachineCreationSucceeded
status: "True"
type: MachineCreation
instanceId: i-0xi5mkzdhi83bmvep8ql
instanceState: Running
metadata: {}
liuhuali@Lius-MacBook-Pro huali-test %
3. delete machine and machine stuck in deleting phase
liuhuali@Lius-MacBook-Pro huali-test % oc delete machine huliu-046-kxqvn-test-9c6lv
machine.machine.openshift.io "huliu-046-kxqvn-test-9c6lv" deleted
^C
liuhuali@Lius-MacBook-Pro huali-test % oc get machine
NAME PHASE TYPE REGION ZONE AGE
huliu-046-kxqvn-drain-4nsb5 Running ecs.g6.large us-east-1 us-east-1a 7m50s
huliu-046-kxqvn-master-0 Running ecs.g6.xlarge us-east-1 us-east-1b 63m
huliu-046-kxqvn-master-1 Running ecs.g6.xlarge us-east-1 us-east-1a 63m
huliu-046-kxqvn-master-2 Running ecs.g6.xlarge us-east-1 us-east-1b 63m
huliu-046-kxqvn-test-9c6lv Deleting ecs.g6.large us-east-1 us-east-1a 14m
huliu-046-kxqvn-test-mx4mh Running ecs.g6.large us-east-1 us-east-1a 6m17s
huliu-046-kxqvn-worker-us-east-1a-qbpck Running ecs.g6.large us-east-1 us-east-1a 57m
huliu-046-kxqvn-worker-us-east-1b-8bp24 Running ecs.g6.large us-east-1 us-east-1b 57m
huliu-046-kxqvn-worker-us-east-1b-f2fnt Running ecs.g6.large us-east-1 us-east-1b 57m
liuhuali@Lius-MacBook-Pro huali-test %
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:0056 |
Description of problem: Machine Deletion Hook cannot work on AliCloud, lifecycleHooks field exists in machine, but machine can be deleted successfully Version-Release number of selected component (if applicable): 4.10.0-0.nightly-2022-01-17-223655 How reproducible: Always Steps to Reproduce: 1.Create a machineset with Machine Deletion Hook, for example spec: lifecycleHooks: preTerminate: - name: MigrateImportantApp owner: etcd 2.Machine created with Machine Deletion Hook liuhuali@Lius-MacBook-Pro huali-test % oc create -f ms3.yaml machineset.machine.openshift.io/huliu-038-fsbld-test created liuhuali@Lius-MacBook-Pro huali-test % oc get machine huliu-038-fsbld-test-b9jwz -o yaml apiVersion: machine.openshift.io/v1beta1 kind: Machine metadata: annotations: machine.openshift.io/instance-state: Running creationTimestamp: "2022-01-19T09:41:00Z" finalizers: - machine.machine.openshift.io generateName: huliu-038-fsbld-test- generation: 2 labels: machine.openshift.io/cluster-api-cluster: huliu-038-fsbld machine.openshift.io/cluster-api-machine-role: worker machine.openshift.io/cluster-api-machine-type: worker machine.openshift.io/cluster-api-machineset: huliu-038-fsbld-test machine.openshift.io/instance-type: ecs.g6.large machine.openshift.io/region: us-east-1 machine.openshift.io/zone: us-east-1a name: huliu-038-fsbld-test-b9jwz namespace: openshift-machine-api ownerReferences: - apiVersion: machine.openshift.io/v1beta1 blockOwnerDeletion: true controller: true kind: MachineSet name: huliu-038-fsbld-test uid: 504e2f1f-7cf3-4bb8-b0ac-407b912dab19 resourceVersion: "97854" uid: ebe33d06-26a0-4890-848c-36bf2dfcd2f5 spec: lifecycleHooks: preTerminate: - name: MigrateImportantApp owner: etcd metadata: {} providerID: alicloud://us-east-1.i-0xih4n0v86yytlhs9giu providerSpec: value: apiVersion: alibabacloudmachineproviderconfig.openshift.io/v1beta1 bandwidth: {} credentialsSecret: name: alibabacloud-credentials dedicatedHostId: "" imageId: m-0xi5sh1y90lsbo8ayl0a instanceType: ecs.g6.large kind: AlibabaCloudMachineProviderConfig metadata: creationTimestamp: null ramRoleName: huliu-038-fsbld-role-worker regionId: us-east-1 securityGroups: - tags: - Key: kubernetes.io/cluster/huliu-038-fsbld Value: owned - Key: GISV Value: ocp - Key: sigs.k8s.io/cloud-provider-alibaba/origin Value: ocp - Key: Name Value: huliu-038-fsbld-sg-worker subscription: {} systemDisk: category: cloud_essd size: 120 tags: - Key: kubernetes.io/cluster/huliu-038-fsbld Value: owned - Key: GISV Value: ocp - Key: sigs.k8s.io/cloud-provider-alibaba/origin Value: ocp userDataSecret: name: worker-user-data vSwitch: tags: - Key: kubernetes.io/cluster/huliu-038-fsbld Value: owned - Key: GISV Value: ocp - Key: sigs.k8s.io/cloud-provider-alibaba/origin Value: ocp - Key: Name Value: huliu-038-fsbld-vswitch-us-east-1a zoneId: us-east-1a status: addresses: - address: 10.0.31.75 type: InternalIP - address: huliu-038-fsbld-test-b9jwz type: InternalDNS conditions: - lastTransitionTime: "2022-01-19T09:42:39Z" status: "True" type: InstanceExists lastUpdated: "2022-01-19T09:46:06Z" nodeRef: kind: Node name: huliu-038-fsbld-test-b9jwz uid: 47359d20-c1b8-48e5-b41f-11698c7bd832 phase: Running providerStatus: conditions: - lastTransitionTime: "2022-01-19T09:42:33Z" message: Machine successfully created reason: MachineCreationSucceeded status: "True" type: MachineCreation instanceId: i-0xih4n0v86yytlhs9giu instanceState: Running metadata: {} liuhuali@Lius-MacBook-Pro huali-test % 3. Machine can be deleted successfully liuhuali@Lius-MacBook-Pro huali-test % oc delete machine huliu-038-fsbld-test-b9jwz machine.machine.openshift.io "huliu-038-fsbld-test-b9jwz" deleted liuhuali@Lius-MacBook-Pro huali-test % oc get machine NAME PHASE TYPE REGION ZONE AGE huliu-038-fsbld-master-0 Running ecs.g6.xlarge us-east-1 us-east-1b 5h huliu-038-fsbld-master-1 Running ecs.g6.xlarge us-east-1 us-east-1a 5h huliu-038-fsbld-master-2 Running ecs.g6.xlarge us-east-1 us-east-1b 5h huliu-038-fsbld-test-p95qb Provisioned ecs.g6.large us-east-1 us-east-1a 2m16s huliu-038-fsbld-worker-us-east-1a-t7kqp Running ecs.g6.large us-east-1 us-east-1a 4h53m huliu-038-fsbld-worker-us-east-1b-gttws Running ecs.g6.large us-east-1 us-east-1b 4h53m huliu-038-fsbld-worker-us-east-1b-xx4pt Running ecs.g6.large us-east-1 us-east-1b 4h53m liuhuali@Lius-MacBook-Pro huali-test % Actual results: Machine deleted successfully Expected results: Machine stuck in deleting phase Additional info: https://issues.redhat.com/browse/OCPCLOUD-1348