Hide Forgot
Description of problem: After creating a new ClusterIngress object, the expected Route53 and ELB objects are created in the AWS account. After deleting the ClusterIngress object, the AWS pieces are not cleaned up. Version-Release number of selected component (if applicable): registry.svc.ci.openshift.org/openshift/origin-v4.0-2019-02-26-105449 How reproducible: 100% Steps to Reproduce: 1. Create clusteringress object apiVersion: ingress.openshift.io/v1alpha1 kind: ClusterIngress metadata: name: myingress namespace: openshift-ingress-operator spec: defaultCertificateSecret: null highAvailability: type: Cloud ingressDomain: myapps.jdiaz-master.new-installer.openshift.com namespaceSelector: null nodePlacement: nodeSelector: matchLabels: node-role.kubernetes.io/worker: "" replicas: 1 routeSelector: null unsupportedExtensions: null 2. Check that AWS objects created successfully 3. Delete ClusterIngress object Actual results: AWS route53 entries, and ELBs created as a result of the ClusterIngress object persist after removing the ClusterIngress object. Expected results: AWS objects get cleaned up. Additional info: Here's the output from the above steps: CREATE: [jdiaz@minigoomba ~]$ oc create -f ~/ingress.yaml clusteringress.ingress.openshift.io/myingress created [jdiaz@minigoomba ~]$ oc get clusteringress myingress -o yaml apiVersion: ingress.openshift.io/v1alpha1 kind: ClusterIngress metadata: creationTimestamp: 2019-02-26T19:35:56Z generation: 1 name: myingress namespace: openshift-ingress-operator resourceVersion: "175204" selfLink: /apis/ingress.openshift.io/v1alpha1/namespaces/openshift-ingress-operator/clusteringresses/myingress uid: bc96e96d-39fd-11e9-ba80-0e9e31d5740e spec: defaultCertificateSecret: null highAvailability: type: Cloud ingressDomain: myapps.jdiaz-master.new-installer.openshift.com namespaceSelector: null nodePlacement: nodeSelector: matchLabels: node-role.kubernetes.io/worker: "" replicas: 1 routeSelector: null unsupportedExtensions: null status: highAvailability: type: Cloud ingressDomain: myapps.jdiaz-master.new-installer.openshift.com labelSelector: app=router,router=router-myingress replicas: 1 QUERY AWS: [jdiaz@minigoomba ~]$ aws route53 list-resource-record-sets --hosted-zone-id Z1QIFLK9U9F5AZ | grep -A7 -B1 myapps { "Name": "\\052.myapps.jdiaz-master.new-installer.openshift.com.", "Type": "A", "AliasTarget": { "HostedZoneId": "Z35SXDOTRQ7X7K", "DNSName": "abca8ff0439fd11e9ba800e9e31d5740-1286579122.us-east-1.elb.amazonaws.com.", "EvaluateTargetHealth": false } } [jdiaz@minigoomba ~]$ aws elb describe-load-balancers --region us-east-1 --load-balancer-names abca8ff0439fd11e9ba800e9e31d5740 | grep DNSName "DNSName": "abca8ff0439fd11e9ba800e9e31d5740-1286579122.us-east-1.elb.amazonaws.com", DELETE: [jdiaz@minigoomba ~]$ oc delete clusteringress myingress clusteringress.ingress.openshift.io "myingress" deleted [jdiaz@minigoomba ~]$ oc get clusteringress myingress -o yaml Error from server (NotFound): clusteringresses.ingress.openshift.io "myingress" not found [jdiaz@minigoomba ~]$ aws route53 list-resource-record-sets --hosted-zone-id Z1QIFLK9U9F5AZ | grep -A7 -B1 myapps { "Name": "\\052.myapps.jdiaz-master.new-installer.openshift.com.", "Type": "A", "AliasTarget": { "HostedZoneId": "Z35SXDOTRQ7X7K", "DNSName": "abca8ff0439fd11e9ba800e9e31d5740-1286579122.us-east-1.elb.amazonaws.com.", "EvaluateTargetHealth": false } } [jdiaz@minigoomba ~]$ aws elb describe-load-balancers --region us-east-1 --load-balancer-names abca8ff0439fd11e9ba800e9e31d5740 | grep DNSName "DNSName": "abca8ff0439fd11e9ba800e9e31d5740-1286579122.us-east-1.elb.amazonaws.com",
might be duplicated to https://bugzilla.redhat.com/show_bug.cgi?id=1682922 please check if the deployment and svc in "openshift-ingress" namespace was deleted when you deleting your clusteringress.
(In reply to hongli from comment #1) > might be duplicated to https://bugzilla.redhat.com/show_bug.cgi?id=1682922 > please check if the deployment and svc in "openshift-ingress" namespace was > deleted when you deleting your clusteringress. Yes, the referenced bugzilla covers the kubernetes objects being left around, and this BZ covers the cloud pieces, but they are effectively the same issue being reported as I also see the in-cluster objects being left around after deleting the clusteringress object.
https://github.com/openshift/cluster-ingress-operator/commit/8246fb3e170ae6796ab9bcd852dc22cf9609b9a8 changed the api to ingresscontrollers. Here is an updated example manifest: $ cat test0-ing.yaml kind: IngressController apiVersion: operator.openshift.io/v1 metadata: name: test0 namespace: openshift-ingress-operator spec: domain: tests0.<your_ingress_domain>
verified with 4.0.0-0.nightly-2019-03-13-233958 and the issue has been fixed. The LB and DNS record related has been removed after ingresscontroller is deleted.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0758