Bug 2058699

Summary: cluster-ingress-operator should report Unupgradeable if user has modified the aws resources annotations
Product: OpenShift Container Platform Reporter: OpenShift BugZilla Robot <openshift-bugzilla-robot>
Component: NetworkingAssignee: Arjun Naik <anaik>
Networking sub component: router QA Contact: Arvind iyengar <aiyengar>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: unspecified CC: aos-bugs, hongli, imm, mjoseph, mmasters, wking
Version: 4.10   
Target Milestone: ---   
Target Release: 4.9.z   
Hardware: All   
OS: All   
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-03-16 11:39:24 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 2039339    
Bug Blocks:    

Comment 3 Arvind iyengar 2022-03-02 08:54:59 UTC
Verified in "4.9.0-0.nightly-2022-03-01-191847" release payload. With this image used for cluster setup,  it is observed that with loadbalancer with "service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags" service level annotation, the ingress operator and the CO resource logs the warning about the presence of the annotation and marks upgradeable=false state:
oc get clusterversion               
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.9.0-0.nightly-2022-03-01-191847   True        False         15m     Cluster version is 4.9.0-0.nightly-2022-03-01-191847

oc -n openshift-ingress annotate service/router-default service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags='test-cluster-qe'
service/router-default annotated

oc -n openshift-ingress get service/router-default -o yaml  
apiVersion: v1
kind: Service
    service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags: test-cluster-qe <--------
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-healthy-threshold: "2"
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-interval: "5"
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-timeout: "4"
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-unhealthy-threshold: "2"
    service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: '*'
    traffic-policy.network.alpha.openshift.io/local-with-fallback: ""
  creationTimestamp: "2022-03-02T08:23:15Z"

oc get co ingress -o yaml 
  - lastTransitionTime: "2022-03-02T08:24:17Z"
    message: desired and current number of IngressControllers are equal
    reason: AsExpected
    status: "False"
    type: Progressing
  - lastTransitionTime: "2022-03-02T08:28:32Z"
    message: The "default" ingress controller reports Degraded=False.
    reason: IngressNotDegraded
    status: "False"
    type: Degraded
  - lastTransitionTime: "2022-03-02T08:49:10Z"
    message: 'Some ingresscontrollers are not upgradeable: ingresscontroller "default"
      is not upgradeable: OperandsNotUpgradeable: One or more managed resources are
      not upgradeable: load balancer service has been modified; changes must be reverted
      before upgrading: '
    reason: IngressControllersNotUpgradeable
    status: "False"
    type: Upgradeable
  extension: null

Comment 6 errata-xmlrpc 2022-03-16 11:39:24 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.9.24 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.