+++ This bug was initially created as a clone of Bug #1825354 +++ In OpenShift 4.4, we enabled HTTP/2 on the frontend for the ingress controller; that is, we allow clients to ask the ingress controller to use HTTP/2, using ALPN. Because the console and oauth-server are both behind the ingress controller's balancer and are using the same serving certificate (namely the ingress controller's default certificate), a browser might perform connection coalescing[1], meaning the browser re-uses the connection that it used to connect to the console route to connect to the oauth route. Because we enabled HTTP/2 on the frontend, this means the browser may connect to the console route using HTTP/2 and then re-use the HTTP/2 connection to try to connect to the oauth route, which fails. In general, we cannot support HTTP/2 ALPN on routes that use the default certificate without risk of connection re-use/coalescing causing problems of this nature. To unblock this issue, we can disable HTTP/2 on the frontend. Later on, in order to support HTTP/2, we will need a solution that enables HTTP/2 only for routes that have custom certificates (which should prevent browsers from coalescing connections). 1. https://daniel.haxx.se/blog/2016/08/18/http2-connection-coalescing/
*** Bug 1826992 has been marked as a duplicate of this bug. ***
Miciah, Can you please answer the questions outlined on the bug this was cloned from, see https://bugzilla.redhat.com/show_bug.cgi?id=1825354#c3
Who is impacted? Customers running 4.5. What is the impact? Browsers that use HTTP/2 and perform aggressive connection coalescing may re-use a previous connection to one route to connect to a different route if the two routes use the same certificate. In particular, this may break the OAuth flow from OpenShift Console, which both use the default certificate. How involved is remediation? A user can wait 30 seconds for keepalive connections to timeout and then retry, or use a browser that does not perform aggressive connection coalescing. Is this a regression? Yes, the issue is caused by HTTP/2 support that was enabled in 4.4.
Sorry, I mixed up my Bugzilla reports (we swapped target releases on this bug and bug 1825354 after I opened this one). "Customers running 4.5" should be replaced with "Customers running 4.4". Following is the corrected text: Who is impacted? Customers running 4.4. What is the impact? Browsers that use HTTP/2 and perform aggressive connection coalescing may re-use a previous connection to one route to connect to a different route if the two routes use the same certificate. In particular, this may break the OAuth flow from OpenShift Console, which both use the default certificate. How involved is remediation? A user can wait 30 seconds for keepalive connections to timeout and then retry, or use a browser that does not perform aggressive connection coalescing. Is this a regression? Yes, the issue is caused by HTTP/2 support that was enabled in 4.4.
Verified with upgrade from 4.3.12 to 4.4.0-0.nightly-2020-04-25-191512 and issue has been fixed. follow below steps and the console, prometheus, grafana UI are all accessible after upgrade. Steps to Reproduce: 1. Create 4.3.12 cluster. 2. Apply OAuth templates 3. Wait for authentication to finish processing the oauth changes 4. Remove OAuth templates 5. Wait for authentication to finish processing the oauth changes 6. Upgrade to 4.4.0-0.nightly-2020-04-25-191512 And the routes are still using HTTP/1.1 after upgrade: $ curl https://console-openshift-console.apps.hongli-bv.qe.devcluster.openshift.com -k -I HTTP/1.1 200 OK $ curl https://oauth-openshift.apps.hongli-bv.qe.devcluster.openshift.com -k -I HTTP/1.1 403 Forbidden
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0581