Bug 1661394 - Unable to authenticate the request due to an error: [invalid bearer token errors in the master logs [NEEDINFO]
Summary: Unable to authenticate the request due to an error: [invalid bearer token err...
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Auth
Version: 3.9.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 3.9.z
Assignee: Stefan Schimanski
QA Contact: Wei Sun
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-12-21 06:06 UTC by Jaspreet Kaur
Modified: 2019-11-20 18:52 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-11-20 18:52:12 UTC
Target Upstream Version:
slaznick: needinfo? (ssadhale)


Attachments (Terms of Use)

Description Jaspreet Kaur 2018-12-21 06:06:50 UTC
Description of problem: Master logs populated with below errors: 


> Dec 11 18:43:23 master.example.com atomic-openshift-master-api[94560]: E1211 18:43:23.469811   94560 authentication.go:64] Unable to authenticate the request due to an error: [invalid bearer token, [invalid bearer token, oauthaccesstokens.oauth.openshift.io "YEx9fyVdKZHo-N3Q0fqtCRBJAa2EkpGlrbujYydRhXo" not found]]

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results: Lots of error messages in the master logs


Expected results: These messages should not be there.


Additional info:

Comment 15 Standa Laznicka 2019-05-29 11:07:12 UTC
Neelesh never really got the answer to his question - what's broken? It's hard to tell what the root cause might be since the only thing we seem to have is the token which can be used by, well, anything.

Comment 16 Shah Zobair 2019-09-17 15:47:56 UTC
Is there a way to track down the requester? We are also facing a similar issue which is generating huge calls to one of the Master node:

Sep 17 11:34:22 ocbdllmcsmst002.devhcloud.XXXX.net atomic-openshift-master-api[68844]: E0917 11:34:22.200563   68844 authentication.go:58] Unable to authenticate the request due to an error: [x509: certificate has expired or is not yet valid, x509: certificate has expired or is not yet valid]
Sep 17 11:34:22 ocbdllmcsmst002.devhcloud.XXXX.net atomic-openshift-master-api[68844]: E0917 11:34:22.205312   68844 authentication.go:58] Unable to authenticate the request due to an error: [x509: certificate has expired or is not yet valid, x509: certificate has expired or is not yet valid]
Sep 17 11:34:22 ocbdllmcsmst002.devhcloud.XXXX.net atomic-openshift-master-api[68844]: E0917 11:34:22.205338   68844 authentication.go:58] Unable to authenticate the request due to an error: [x509: certificate has expired or is not yet valid, x509: certificate has expired or is not yet valid]
Sep 17 11:34:22 ocbdllmcsmst002.devhcloud.XXXX.net atomic-openshift-master-api[68844]: E0917 11:34:22.205339   68844 authentication.go:58] Unable to authenticate the request due to an error: [x509: certificate has expired or is not yet valid, x509: certificate has expired or is not yet valid]
Sep 17 11:34:22 ocbdllmcsmst002.devhcloud.XXXX.net atomic-openshift-master-api[68844]: E0917 11:34:22.206694   68844 authentication.go:58] Unable to authenticate the request due to an error: [x509: certificate has expired or is not yet valid, x509: certificate has expired or is not yet valid]
Sep 17 11:34:22 ocbdllmcsmst002.devhcloud.XXXX.net atomic-openshift-master-api[68844]: E0917 11:34:22.206766   68844 authentication.go:58] Unable to authenticate the request due to an error: [x509: certificate has expired or is not yet valid, x509: certificate has expired or is not yet valid]
Sep 17 11:34:22 ocbdllmcsmst002.devhcloud.XXXX.net atomic-openshift-master-api[68844]: E0917 11:34:22.544204   68844 authentication.go:58] Unable to authenticate the request due to an error: [x509: certificate has expired or is not yet valid, x509: certificate has expired or is not yet valid]

Comment 17 Standa Laznicka 2019-09-18 06:35:36 UTC
That's a completely different issue - one of your certificates expired. You may want to check the logs, specifically the audit log, and renew the certificate for whoever is the caller.

Comment 19 Shah Zobair 2019-09-18 13:55:47 UTC
I have checked all the certificates and those were recently renewed. Here is the certificate validity report: https://access.redhat.com/hydra/rest/cases/02463245/attachments/137694fe-70aa-4049-a1b1-67512b5a7a36 . There might be something else that we are missing, but can't identify the component from the logs.

Thanks

Comment 21 Stephen Cuppett 2019-11-20 18:52:12 UTC
OCP 3.6-3.10 is no longer on full support [1]. Marking CLOSED DEFERRED. If you have a customer case with a support exception or have reproduced on 3.11+, please reopen and include those details. When reopening, please set the Target Release to the appropriate version where needed.

[1]: https://access.redhat.com/support/policy/updates/openshift


Note You need to log in before you can comment on or make changes to this bug.