Bug 1932649

Summary:	Cluster Ingress Operator degrades if external LB redirects http to https because of new "canary" route
Product:	OpenShift Container Platform	Reporter:	Stephen Greene <sgreene>
Component:	Networking	Assignee:	Stephen Greene <sgreene>
Networking sub component:	router	QA Contact:	Hongan Li <hongli>
Status:	CLOSED ERRATA	Docs Contact:
Severity:	urgent
Priority:	urgent	CC:	aos-bugs, bbennett, hongli, josef.meier, mjoseph, scuppett, sgreene, sponnaga
Version:	4.7	Keywords:	Upgrades
Target Milestone:	---
Target Release:	4.7.z
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	Cause: Exposing the default ingress controller via an external load balancer that redirects all HTTP traffic to HTTPS Consequence: Ingress Canary endpoint checks performed by the ingress operator would fail, which would ultimately cause the ingress cluster operator to become degraded. Fix: Convert the cleartext canary route to an edge encrypted route. Result: The canary route works via HTTPS only load balancers, when insecure traffic is redirected by the load balancer.	Story Points:	---
Clone Of:	1932401	Environment:
Last Closed:	2021-03-10 11:24:00 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1932401
Bug Blocks:

Description Stephen Greene 2021-02-24 20:26:44 UTC

+++ This bug was initially created as a clone of Bug #1932401 +++

Hi,

in my company we use an external load balancer that redirects HTTP traffic to HTTPS.

During an upgrade from 4.6 to 4.7 the cluster-ingress-operator degraded because it couldn't reach the new canary route in openshift-ingress-canary.

I saw that this canary route is a HTTP route. This won't work in our setup.

I manually added edge termination to this route and immediately the upgrade proceeded.

This is a PR that should add 'edge' termination to the canary route:
https://github.com/openshift/cluster-ingress-operator/pull/555

Thanks and regards,

Josef

Comment 3 Hongan Li 2021-03-04 04:01:19 UTC

verified with 4.7.0-0.nightly-2021-03-04-004412 and passed.

$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.7.0-0.nightly-2021-03-04-004412   True        False         88m     Cluster version is 4.7.0-0.nightly-2021-03-04-004412

$ oc -n openshift-ingress-canary get route
NAME     HOST/PORT                                                                            PATH   SERVICES         PORT   TERMINATION     WILDCARD
canary   canary-openshift-ingress-canary.apps.hongli-47bv.qe.azure.devcluster.openshift.com          ingress-canary   8080   edge/Redirect   None

$ curl -k https://canary-openshift-ingress-canary.apps.hongli-47bv.qe.azure.devcluster.openshift.com 
Hello OpenShift!

$ curl -kL http://canary-openshift-ingress-canary.apps.hongli-47bv.qe.azure.devcluster.openshift.com 
Hello OpenShift!

Comment 6 errata-xmlrpc 2021-03-10 11:24:00 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.7.1 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:0678