Description of problem: Master logs populated with below errors: > Dec 11 18:43:23 master.example.com atomic-openshift-master-api[94560]: E1211 18:43:23.469811 94560 authentication.go:64] Unable to authenticate the request due to an error: [invalid bearer token, [invalid bearer token, oauthaccesstokens.oauth.openshift.io "YEx9fyVdKZHo-N3Q0fqtCRBJAa2EkpGlrbujYydRhXo" not found]] Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Lots of error messages in the master logs Expected results: These messages should not be there. Additional info:
Neelesh never really got the answer to his question - what's broken? It's hard to tell what the root cause might be since the only thing we seem to have is the token which can be used by, well, anything.
Is there a way to track down the requester? We are also facing a similar issue which is generating huge calls to one of the Master node: Sep 17 11:34:22 ocbdllmcsmst002.devhcloud.XXXX.net atomic-openshift-master-api[68844]: E0917 11:34:22.200563 68844 authentication.go:58] Unable to authenticate the request due to an error: [x509: certificate has expired or is not yet valid, x509: certificate has expired or is not yet valid] Sep 17 11:34:22 ocbdllmcsmst002.devhcloud.XXXX.net atomic-openshift-master-api[68844]: E0917 11:34:22.205312 68844 authentication.go:58] Unable to authenticate the request due to an error: [x509: certificate has expired or is not yet valid, x509: certificate has expired or is not yet valid] Sep 17 11:34:22 ocbdllmcsmst002.devhcloud.XXXX.net atomic-openshift-master-api[68844]: E0917 11:34:22.205338 68844 authentication.go:58] Unable to authenticate the request due to an error: [x509: certificate has expired or is not yet valid, x509: certificate has expired or is not yet valid] Sep 17 11:34:22 ocbdllmcsmst002.devhcloud.XXXX.net atomic-openshift-master-api[68844]: E0917 11:34:22.205339 68844 authentication.go:58] Unable to authenticate the request due to an error: [x509: certificate has expired or is not yet valid, x509: certificate has expired or is not yet valid] Sep 17 11:34:22 ocbdllmcsmst002.devhcloud.XXXX.net atomic-openshift-master-api[68844]: E0917 11:34:22.206694 68844 authentication.go:58] Unable to authenticate the request due to an error: [x509: certificate has expired or is not yet valid, x509: certificate has expired or is not yet valid] Sep 17 11:34:22 ocbdllmcsmst002.devhcloud.XXXX.net atomic-openshift-master-api[68844]: E0917 11:34:22.206766 68844 authentication.go:58] Unable to authenticate the request due to an error: [x509: certificate has expired or is not yet valid, x509: certificate has expired or is not yet valid] Sep 17 11:34:22 ocbdllmcsmst002.devhcloud.XXXX.net atomic-openshift-master-api[68844]: E0917 11:34:22.544204 68844 authentication.go:58] Unable to authenticate the request due to an error: [x509: certificate has expired or is not yet valid, x509: certificate has expired or is not yet valid]
That's a completely different issue - one of your certificates expired. You may want to check the logs, specifically the audit log, and renew the certificate for whoever is the caller.
I have checked all the certificates and those were recently renewed. Here is the certificate validity report: https://access.redhat.com/hydra/rest/cases/02463245/attachments/137694fe-70aa-4049-a1b1-67512b5a7a36 . There might be something else that we are missing, but can't identify the component from the logs. Thanks
OCP 3.6-3.10 is no longer on full support [1]. Marking CLOSED DEFERRED. If you have a customer case with a support exception or have reproduced on 3.11+, please reopen and include those details. When reopening, please set the Target Release to the appropriate version where needed. [1]: https://access.redhat.com/support/policy/updates/openshift