Bug 1905573

Summary: [4.6] Changing the bound token service account issuer invalids previously issued bound tokens
Product: OpenShift Container Platform Reporter: Maru Newby <mnewby>
Component: kube-apiserverAssignee: Maru Newby <mnewby>
Status: CLOSED ERRATA QA Contact: scheng
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.6CC: aos-bugs, mfojtik, scheng, xxia
Target Milestone: ---   
Target Release: 4.6.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Known Issue
Doc Text:
Cause: Changing the serviceAccountIssuer field of the authentication resource will update the kube-apiserver to validate tokens with the new issuer and reject tokens with the previous issuer. kube-apiserver does not support multiple issuers at this time, so a graceful transition is not possible. Consequence: Changing the serviceAccountIssuer has the potential to disrupt applications relying on bound tokens. Unless an application is coded to explicitly request a new token when their existing token starts receiving 401 responses from the apiserver, they will continue to use the invalid token until restarted or until their invalid token exceeds 80% of its duration (at which point the kubelet will request a new token for them). Workaround (if any): Only change the serviceAccountIssuer field if disruption is acceptable and restarting all pods is an option.
 Story Points: ---
Clone Of: 1905328
: 1905576 (view as bug list) Environment:
Last Closed: 2021-02-08 13:50:52 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1905328    
Bug Blocks: 1905576    

Description Maru Newby 2020-12-08 15:16:41 UTC 
+++ This bug was initially created as a clone of Bug #1905328 +++

Changing the serviceAccountIssuer field of the authentication resource will update the kube-apiserver to validate tokens with the new issuer and reject tokens with the previous issuer. This has the potential to disrupt applications relying on bound tokens. Unless an application is coded to explicitly request a new token when their existing token starts receiving 401 responses from the apiserver, they will continue to use the invalid token until restarted or until their invalid token exceeds 80% of its duration (at which point the kubelet will request a new token for them).

A likely fix for 4.8 is ensuring the apiserver supports a graceful transition between 2 issuers. For 4.7, 4.6, and 4.5, the near-term remedy is to document the impact of a change in issuer and ensure the compatibility of control plane components like controller manager with an issuer change.

--- Additional comment from OpenShift Automated Release Tooling on 2020-12-08 15:02:55 UTC ---

Elliott changed bug status from MODIFIED to ON_QA.
Comment 1 Maru Newby 2021-01-18 14:56:21 UTC 
Bumping to high. The risk of this change is zero - doc only - and changing the issuer has the potential to disrupt user workloads by invalidating all previously issued tokens.
Comment 8 errata-xmlrpc 2021-02-08 13:50:52 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Container Platform 4.6.16 security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:0308