Hide Forgot
Description of problem: The cio creates a service of type "LoadBalancer" for every IngressController resource. The user can subsequently modify the annotation "service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags" to specify additional tags on the AWS loadbalancer. When this is the case the operator should report the IngressController as Upgradeable=False and also the ClusterOperator status condition of Upgradeable=False. OpenShift release version: Cluster Platform: How reproducible: Steps to Reproduce (in detail): 1. 2. 3. Actual results: Expected results: Impact of the problem: Additional info: ** Please do not disregard the report template; filling the template out as much as possible will allow us to help you. Please consider attaching a must-gather archive (via `oc adm must-gather`). Please review must-gather contents for sensitive information before attaching any must-gathers to a bugzilla report. You may also mark the bug private if you wish.
Setting blocker- as this isn't a regression or upgrade blocker. Arjun, I notice you have opened https://github.com/openshift/cluster-ingress-operator/pull/693 for this BZ, so I am assigning the BZ to you. We'll need to discuss the mechanics of getting the change in the branch in which we need it.
Verified in "4.10.0-0.nightly-2022-01-22-102609" release version. With this payload, it is observed that with loadbalancer with "service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags" service level annotation, the ingress operator and the CO resource logs the warning about the presence of the annotation and marks upgradeable=false state: ------ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.10.0-0.nightly-2022-01-22-102609 True False 135m Cluster version is 4.10.0-0.nightly-2022-01-22-102609 oc -n openshift-ingress annotate service/router-internalapps service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags='test-cluster-qe' service/router-internalapps annotated oc -n openshift-ingress get service/router-internalapps -o yaml apiVersion: v1 kind: Service metadata: annotations: service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags: test-cluster-qe <----- service.beta.kubernetes.io/aws-load-balancer-healthcheck-healthy-threshold: "2" service.beta.kubernetes.io/aws-load-balancer-healthcheck-interval: "10" service.beta.kubernetes.io/aws-load-balancer-healthcheck-timeout: "4" service.beta.kubernetes.io/aws-load-balancer-healthcheck-unhealthy-threshold: "2" service.beta.kubernetes.io/aws-load-balancer-type: nlb traffic-policy.network.alpha.openshift.io/local-with-fallback: "" creationTimestamp: "2022-01-24T05:26:09Z" check the ingresscontroller and the related CO: oc -n openshift-ingress-operator get ingresscontroller internalapps -o yaml - lastTransitionTime: "2022-01-24T05:26:47Z" status: "False" type: Degraded - lastTransitionTime: "2022-01-24T05:48:14Z" message: 'One or more managed resources are not upgradeable: load balancer service <------ has been modified; changes must be reverted before upgrading: ' reason: OperandsNotUpgradeable status: "False" <------ type: Upgradeable <------ domain: internalapps.pdhamdhe41024.qe.devcluster.openshift.com endpointPublishingStrategy: loadBalancer: providerParameters: aws: type: NLB type: AWS scope: External type: LoadBalancerService oc get co ingress -o yaml - lastTransitionTime: "2022-01-24T03:41:38Z" message: The "default" ingress controller reports Degraded=False. reason: IngressNotDegraded status: "False" type: Degraded - lastTransitionTime: "2022-01-24T05:48:15Z" message: 'Some ingresscontrollers are not upgradeable: ingresscontroller "internalapps" <------ is not upgradeable: OperandsNotUpgradeable: One or more managed resources are not upgradeable: load balancer service has been modified; changes must be reverted before upgrading: ' reason: IngressControllersNotUpgradeable status: "False" <------ type: Upgradeable <------ extension: null ------
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:0056