Bug 2108214

Summary:	Route status isn't always getting cleared with routeSelector updates
Product:	OpenShift Container Platform	Reporter:	OpenShift BugZilla Robot <openshift-bugzilla-robot>
Component:	Networking	Assignee:	Grant Spence <gspence>
Networking sub component:	router	QA Contact:	Melvin Joseph <mjoseph>
Status:	CLOSED ERRATA	Docs Contact:
Severity:	medium
Priority:	medium	CC:	aos-bugs, gspence, hongli, mmasters
Version:	4.11
Target Milestone:	---
Target Release:	4.11.z
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-09-20 16:34:44 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	2101878, 2110528
Bug Blocks:	2024946

Description OpenShift BugZilla Robot 2022-07-18 15:35:58 UTC

+++ This bug was initially created as a clone of Bug #2101878 +++

Description of problem:
https://bugzilla.redhat.com/show_bug.cgi?id=1944851# introduced a fix to clearing route status, however, QA found that it wasn't consistently working.

OpenShift release version:
4.11

Cluster Platform:
All

How reproducible:
25%-50% of the time

Steps to Reproduce (in detail):
1. Create a new cluster
2. Shard the default ingress controller:
   oc -n openshift-ingress-operator patch ingresscontroller/default --type=merge --patch='{"spec":{"routeSelector":{"matchLabels":{"type":"shard"}}}}';
3. Wait about 30-60 seconds
4. Is canary route status cleared?
   oc get route -n openshift-ingress-canary -o yaml

Actual results:
Sometimes, canary/console/auth route don't clear status when sharding the default ingress controller

Expected results:
Status should always clear if the route is no longer admitted.

Impact of the problem:
No impact other than it's confusing to developers to debug route status

Additional info:



** Please do not disregard the report template; filling the template out as much as possible will allow us to help you. Please consider attaching a must-gather archive (via `oc adm must-gather`). Please review must-gather contents for sensitive information before attaching any must-gathers to a bugzilla report.  You may also mark the bug private if you wish.

Comment 1 Melvin Joseph 2022-07-25 05:31:42 UTC

@gspence kindly added https://github.com/openshift/cluster-ingress-operator/pull/811, which i guess is also required?

Comment 5 Melvin Joseph 2022-09-14 05:56:36 UTC

melvinjoseph@mjoseph-mac Downloads % oc get clusterversion
NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.11.0-0.nightly-2022-09-13-152412   True        False         79m     Cluster version is 4.11.0-0.nightly-2022-09-13-152412
melvinjoseph@mjoseph-mac Downloads % oc get route -o json -n openshift-ingress-canary   canary  | jq '.status'
{
  "ingress": [
    {
      "conditions": [
        {
          "lastTransitionTime": "2022-09-14T03:55:51Z",
          "status": "True",
          "type": "Admitted"
        }
      ],
      "host": "canary-openshift-ingress-canary.apps.mjoseph-7043.qe.devcluster.openshift.com",
      "routerCanonicalHostname": "router-default.apps.mjoseph-7043.qe.devcluster.openshift.com",
      "routerName": "default",
      "wildcardPolicy": "None"
    }
  ]
}
melvinjoseph@mjoseph-mac Downloads % oc -n openshift-ingress get pods
oNAME                              READY   STATUS    RESTARTS       AGE
router-default-76877957bb-42vbq   1/1     Running   2 (101m ago)   109m
router-default-76877957bb-dz57f   1/1     Running   0              98m
melvinjoseph@mjoseph-mac Downloads % oc -n openshift-ingress-operator patch ingresscontroller/default --type=merge --patch='{"spec":{"routeSelector":{"matchLabels":{"type":"shard"}}}}';
ingresscontroller.operator.openshift.io/default patched
melvinjoseph@mjoseph-mac Downloads % oc -n openshift-ingress get pods                                                                                                  




NAME                              READY   STATUS        RESTARTS       AGE
router-default-5dd87b9449-8hmgd   1/1     Running       0              11s
router-default-5dd87b9449-kkslj   1/1     Running       0              10s
router-default-76877957bb-42vbq   1/1     Terminating   2 (101m ago)   109m
router-default-76877957bb-dz57f   1/1     Running       0              98m
melvinjoseph@mjoseph-mac Downloads % 
melvinjoseph@mjoseph-mac Downloads % 
melvinjoseph@mjoseph-mac Downloads % 
melvinjoseph@mjoseph-mac Downloads % 
melvinjoseph@mjoseph-mac Downloads % oc get route -o json -n openshift-ingress-canary   canary  | jq '.status'                                                                           
{
  "ingress": [
    {
      "conditions": [
        {
          "lastTransitionTime": "2022-09-14T03:55:51Z",
          "status": "True",
          "type": "Admitted"
        }
      ],
      "host": "canary-openshift-ingress-canary.apps.mjoseph-7043.qe.devcluster.openshift.com",
      "routerCanonicalHostname": "router-default.apps.mjoseph-7043.qe.devcluster.openshift.com",
      "routerName": "default",
      "wildcardPolicy": "None"
    }
  ]
}
melvinjoseph@mjoseph-mac Downloads % oc -n openshift-ingress get pods                                         
NAME                              READY   STATUS        RESTARTS   AGE
router-default-5dd87b9449-8hmgd   1/1     Running       0          78s
router-default-5dd87b9449-kkslj   1/1     Running       0          77s
router-default-76877957bb-dz57f   0/1     Terminating   0          99m
melvinjoseph@mjoseph-mac Downloads % oc get route -o json -n openshift-ingress-canary   canary  | jq '.status'
{}
melvinjoseph@mjoseph-mac Downloads % oc get route -A 
NAMESPACE                  NAME                      HOST/PORT                                                                                    PATH        SERVICES            PORT    TERMINATION            WILDCARD
openshift-authentication   oauth-openshift           oauth-openshift.apps.mjoseph-7043.qe.devcluster.openshift.com                                            oauth-openshift     6443    passthrough/Redirect   None
openshift-console          console                   console-openshift-console.apps.mjoseph-7043.qe.devcluster.openshift.com                                  console             https   reencrypt/Redirect     None
openshift-console          downloads                 downloads-openshift-console.apps.mjoseph-7043.qe.devcluster.openshift.com                                downloads           http    edge/Redirect          None
openshift-ingress-canary   canary                    canary-openshift-ingress-canary.apps.mjoseph-7043.qe.devcluster.openshift.com                            ingress-canary      8080    edge/Redirect          None
openshift-monitoring       alertmanager-main         alertmanager-main-openshift-monitoring.apps.mjoseph-7043.qe.devcluster.openshift.com         /api        alertmanager-main   web     reencrypt/Redirect     None
openshift-monitoring       prometheus-k8s            prometheus-k8s-openshift-monitoring.apps.mjoseph-7043.qe.devcluster.openshift.com            /api        prometheus-k8s      web     reencrypt/Redirect     None
openshift-monitoring       prometheus-k8s-federate   prometheus-k8s-federate-openshift-monitoring.apps.mjoseph-7043.qe.devcluster.openshift.com   /federate   prometheus-k8s      web     reencrypt/Redirect     None
openshift-monitoring       thanos-querier            thanos-querier-openshift-monitoring.apps.mjoseph-7043.qe.devcluster.openshift.com            /api        thanos-querier      web     reencrypt/Redirect     None
melvinjoseph@mjoseph-mac Downloads % oc get route -o json -n openshift-authentication oauth-openshift   | jq '.status'
{}
melvinjoseph@mjoseph-mac Downloads % oc get route -o json -n openshift-ingress-canary   canary  | jq '.status'  
{}
melvinjoseph@mjoseph-mac Downloads % oc -n openshift-ingress get pods                                         
NAME                              READY   STATUS    RESTARTS   AGE
router-default-5dd87b9449-8hmgd   1/1     Running   0          2m16s
router-default-5dd87b9449-kkslj   1/1     Running   0          2m15s
melvinjoseph@mjoseph-mac Downloads % oc -n openshift-ingress-operator patch ingresscontroller/default --type=merge --patch='{"spec":{"routeSelector":{"matchLabels":{"type":null}}}}';
ingresscontroller.operator.openshift.io/default patched
melvinjoseph@mjoseph-mac Downloads % oc -n openshift-ingress get pods                                                                                                                 
NAME                              READY   STATUS        RESTARTS   AGE
router-default-5dd87b9449-kkslj   1/1     Terminating   0          3m16s
router-default-87dc7c544-q4tj9    1/1     Running       0          56s
router-default-87dc7c544-w59nc    1/1     Running       0          56s
melvinjoseph@mjoseph-mac Downloads % oc get route -o json -n openshift-ingress-canary   canary  | jq '.status'                                                                        
{
  "ingress": [
    {
      "conditions": [
        {
          "lastTransitionTime": "2022-09-14T05:36:28Z",
          "status": "True",
          "type": "Admitted"
        }
      ],
      "host": "canary-openshift-ingress-canary.apps.mjoseph-7043.qe.devcluster.openshift.com",
      "routerCanonicalHostname": "router-default.apps.mjoseph-7043.qe.devcluster.openshift.com",
      "routerName": "default",
      "wildcardPolicy": "None"
    }
  ]
}
melvinjoseph@mjoseph-mac Downloads % oc -n openshift-ingress get pods                                           
NAME                              READY   STATUS        RESTARTS   AGE
router-default-5dd87b9449-kkslj   0/1     Terminating   0          3m51s
router-default-87dc7c544-q4tj9    1/1     Running       0          91s
router-default-87dc7c544-w59nc    1/1     Running       0          91s
melvinjoseph@mjoseph-mac Downloads % oc -n openshift-ingress get pods
NAME                             READY   STATUS    RESTARTS   AGE
router-default-87dc7c544-q4tj9   1/1     Running   0          2m2s
router-default-87dc7c544-w59nc   1/1     Running   0          2m2s

Hence marking as verified

Comment 8 errata-xmlrpc 2022-09-20 16:34:44 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.11.5 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:6536