Bug 2039339 - cluster-ingress-operator should report Unupgradeable if user has modified the aws resources annotations
Summary: cluster-ingress-operator should report Unupgradeable if user has modified the...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.10
Hardware: All
OS: All
unspecified
medium
Target Milestone: ---
: 4.10.0
Assignee: Arjun Naik
QA Contact: Arvind iyengar
URL:
Whiteboard:
Depends On:
Blocks: 2058699
TreeView+ depends on / blocked
 
Reported: 2022-01-11 14:17 UTC by Arjun Naik
Modified: 2023-09-18 04:30 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: The cluster-ingress-operator did not managed the aws resources annotation once the LoadBalancer type Service was created. Users could update the annotation value. Consequence: If the tags in Infrastructure were changed the new values could not be written to the annotation without overwriting the value set the user. Fix: The operator now warns the user that the corresponding IngressController is degraded when the annotation value is out of sync. Result: In subsequent versions of OCP the annotations value can be safely updated by the cluster-ingress-operator based on external sources.
Clone Of:
Environment:
Last Closed: 2022-03-12 04:40:34 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift cluster-ingress-operator pull 693 0 None Merged Bug 2039339: Upgradeable Condition in Operator and IC status 2022-03-16 21:19:08 UTC
Red Hat Product Errata RHSA-2022:0056 0 None None None 2022-03-12 04:40:44 UTC

Description Arjun Naik 2022-01-11 14:17:57 UTC
Description of problem: The cio creates a service of type "LoadBalancer" for every IngressController resource. The user can subsequently modify the annotation "service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags" to specify additional tags on the AWS loadbalancer. When this is the case the operator should report the IngressController as Upgradeable=False and also the ClusterOperator status condition of Upgradeable=False. 


OpenShift release version:


Cluster Platform:


How reproducible:


Steps to Reproduce (in detail):
1.
2.
3.


Actual results:


Expected results:


Impact of the problem:


Additional info:



** Please do not disregard the report template; filling the template out as much as possible will allow us to help you. Please consider attaching a must-gather archive (via `oc adm must-gather`). Please review must-gather contents for sensitive information before attaching any must-gathers to a bugzilla report.  You may also mark the bug private if you wish.

Comment 1 Miciah Dashiel Butler Masters 2022-01-11 17:00:23 UTC
Setting blocker- as this isn't a regression or upgrade blocker.  

Arjun, I notice you have opened https://github.com/openshift/cluster-ingress-operator/pull/693 for this BZ, so I am assigning the BZ to you.  We'll need to discuss the mechanics of getting the change in the branch in which we need it.

Comment 5 Arvind iyengar 2022-01-24 06:05:57 UTC
Verified in "4.10.0-0.nightly-2022-01-22-102609" release version. With this payload, it is observed that with loadbalancer with "service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags" service level annotation, the ingress operator and the CO resource logs the warning about the presence of the annotation and marks upgradeable=false state:
------
oc get clusterversion  
NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.10.0-0.nightly-2022-01-22-102609   True        False         135m    Cluster version is 4.10.0-0.nightly-2022-01-22-102609

oc -n openshift-ingress annotate service/router-internalapps service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags='test-cluster-qe'
service/router-internalapps annotated

oc -n openshift-ingress get service/router-internalapps -o yaml
apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-additional-resource-tags: test-cluster-qe <-----
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-healthy-threshold: "2"
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-interval: "10"
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-timeout: "4"
    service.beta.kubernetes.io/aws-load-balancer-healthcheck-unhealthy-threshold: "2"
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
    traffic-policy.network.alpha.openshift.io/local-with-fallback: ""
  creationTimestamp: "2022-01-24T05:26:09Z"

check the ingresscontroller and the related CO:
oc -n openshift-ingress-operator get ingresscontroller internalapps -o yaml 
  - lastTransitionTime: "2022-01-24T05:26:47Z"
    status: "False"
    type: Degraded
  - lastTransitionTime: "2022-01-24T05:48:14Z"
    message: 'One or more managed resources are not upgradeable: load balancer service <------
      has been modified; changes must be reverted before upgrading: '
    reason: OperandsNotUpgradeable
    status: "False"   <------
    type: Upgradeable <------
  domain: internalapps.pdhamdhe41024.qe.devcluster.openshift.com
  endpointPublishingStrategy:
    loadBalancer:
      providerParameters:
        aws:
          type: NLB
        type: AWS
      scope: External
    type: LoadBalancerService
    

oc get co ingress -o yaml
  - lastTransitionTime: "2022-01-24T03:41:38Z"
    message: The "default" ingress controller reports Degraded=False.
    reason: IngressNotDegraded
    status: "False" 
    type: Degraded
  - lastTransitionTime: "2022-01-24T05:48:15Z"
    message: 'Some ingresscontrollers are not upgradeable: ingresscontroller "internalapps" <------
      is not upgradeable: OperandsNotUpgradeable: One or more managed resources are
      not upgradeable: load balancer service has been modified; changes must be reverted
      before upgrading: '
    reason: IngressControllersNotUpgradeable
    status: "False" <------
    type: Upgradeable <------
  extension: null
------

Comment 8 errata-xmlrpc 2022-03-12 04:40:34 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0056

Comment 11 Red Hat Bugzilla 2023-09-18 04:30:04 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days


Note You need to log in before you can comment on or make changes to this bug.