Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1922961

Summary:	Engine log flooded with OAuthException messages after the upgrade from 4.3 to 4.4.
Product:	Red Hat Enterprise Virtualization Manager	Reporter:	Miguel Martin <mmartinv>
Component:	ovirt-engine	Assignee:	Artur Socha <asocha>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Lucie Leistnerova <lleistne>
Severity:	high	Docs Contact:
Priority:	high
Version:	4.4.3	CC:	amashah, amusil, asocha, emesika, gchakkar, gzaidman, mperina, thomas.mueller2, usurse
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-04-01 13:20:44 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	Infra	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Miguel Martin 2021-02-01 08:02:53 UTC

Description of problem:
After the upgrade from 4.3 to 4.4 the engine log is flooded with OAuthException messages:
~~~
2021-01-20 03:47:12,800+03 INFO  [org.ovirt.engine.core.sso.utils.AuthenticationUtils] (default task-479) [] User admin@internal successfully logged in with scopes: ovirt-app-api ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate ovirt-ext=token:password-access
2021-01-20 03:47:12,820+03 INFO  [org.ovirt.engine.core.sso.utils.AuthenticationUtils] (default task-481) [] User admin@internal successfully logged in with scopes: ovirt-app-api ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate ovirt-ext=token:password-access
2021-01-20 03:47:12,899+03 ERROR [org.ovirt.engine.core.sso.utils.SsoUtils] (default task-479) [] OAuthException invalid_grant: The provided authorization grant for the auth code has expired.
2021-01-20 03:47:12,900+03 ERROR [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-481) [] Cannot authenticate using authentication Headers: invalid_grant: The provided authorization grant for the auth code has expired.
2021-01-20 03:47:12,943+03 INFO  [org.ovirt.engine.core.sso.utils.AuthenticationUtils] (default task-481) [] User admin@internal successfully logged in with scopes: ovirt-app-api ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate ovirt-ext=token:password-access
2021-01-20 03:47:13,110+03 ERROR [org.ovirt.engine.core.sso.utils.SsoUtils] (default task-479) [] OAuthException invalid_grant: The provided authorization grant for the auth code has expired.
2021-01-20 03:47:13,110+03 ERROR [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-481) [] Cannot authenticate using authentication Headers: invalid_grant: The provided authorization grant for the auth code has expired.
2021-01-20 03:47:13,172+03 INFO  [org.ovirt.engine.core.sso.utils.AuthenticationUtils] (default task-481) [] User admin@internal successfully logged in with scopes: ovirt-app-api ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate ovirt-ext=token:password-access
2021-01-20 03:47:13,219+03 ERROR [org.ovirt.engine.core.sso.utils.SsoUtils] (default task-479) [] OAuthException invalid_grant: The provided authorization grant for the auth code has expired.
2021-01-20 03:47:13,219+03 ERROR [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-481) [] Cannot authenticate using authentication Headers: invalid_grant: The provided authorization grant for the auth code has expired.
2021-01-20 03:47:13,255+03 INFO  [org.ovirt.engine.core.sso.utils.AuthenticationUtils] (default task-481) [] User admin@internal successfully logged in with scopes: ovirt-app-api ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate ovirt-ext=token:password-access
2021-01-20 03:47:13,297+03 ERROR [org.ovirt.engine.core.sso.utils.SsoUtils] (default task-479) [] OAuthException invalid_grant: The provided authorization grant for the auth code has expired.
2021-01-20 03:47:13,298+03 ERROR [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-481) [] Cannot authenticate using authentication Headers: invalid_grant: The provided authorization grant for the auth code has expired.
2021-01-20 03:47:13,339+03 INFO  [org.ovirt.engine.core.sso.utils.AuthenticationUtils] (default task-481) [] User admin@internal successfully logged in with scopes: ovirt-app-api ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate ovirt-ext=token:password-access
2021-01-20 03:47:13,460+03 ERROR [org.ovirt.engine.core.sso.utils.SsoUtils] (default task-479) [] OAuthException invalid_grant: The provided authorization grant for the auth code has expired.
2021-01-20 03:47:13,460+03 ERROR [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-481) [] Cannot authenticate using authentication Headers: invalid_grant: The provided authorization grant for the auth code has expired.
2021-01-20 03:47:13,528+03 INFO  [org.ovirt.engine.core.sso.utils.AuthenticationUtils] (default task-481) [] User admin@internal successfully logged in with scopes: ovirt-app-api ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate ovirt-ext=token:password-access
2021-01-20 03:47:13,677+03 ERROR [org.ovirt.engine.core.sso.utils.SsoUtils] (default task-479) [] OAuthException invalid_grant: The provided authorization grant for the auth code has expired.
2021-01-20 03:47:13,678+03 ERROR [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-481) [] Cannot authenticate using authentication Headers: invalid_grant: The provided authorization grant for the auth code has expired.
2021-01-20 03:47:13,684+03 ERROR [org.ovirt.engine.core.sso.utils.SsoUtils] (default task-457) [] OAuthException invalid_grant: The provided authorization grant for the auth code has expired.
2021-01-20 03:47:13,685+03 ERROR [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-447) [] Cannot authenticate using authentication Headers: invalid_grant: The provided authorization grant for the auth code has expired.
2021-01-20 03:47:13,700+03 ERROR [org.ovirt.engine.core.sso.utils.SsoUtils] (default task-457) [] OAuthException invalid_grant: The provided authorization grant for the auth code has expired.
2021-01-20 03:47:13,700+03 ERROR [org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter] (default task-481) [] Cannot authenticate using authentication Headers: invalid_grant: The provided authorization grant for the auth code has expired.
~~~

However, the authentication seems to be working as expected.

Version-Release number of selected component (if applicable):
4.4.3

How reproducible:
In CU environment

Comment 1 Martin Perina 2021-02-01 09:28:44 UTC

Haven't you recently changed password for admin@internal user? If so, do you have OVN provider configured (or any other monitoring solution accessing RHV Manager using RESTAPI? If so, have changed the password for admin@internal?

Below is a link how to change a password for admin@internal and how to update OVN provider:

https://access.redhat.com/solutions/63677
https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.4/html/administration_guide/sect-adding_external_providers#Adding_OVN_as_an_External_Network_Provider

Comment 2 Miguel Martin 2021-02-01 10:18:40 UTC

Sorry, I pasted the messages related to the 'admin@internal' user only but it looks like it also happens with LDAP users.

Either way, I will ask the customers if they changed the admin password recently.

Comment 16 Martin Perina 2021-02-12 16:18:45 UTC

This issue cannot be fixed on RHV side, it need to be fixed on OCP/CSI side as tracked on BZ1924626 and BZ1924623.

Comment 17 Martin Perina 2021-04-01 13:20:44 UTC

Closing as currentrelease, because both dependent (BZ1924626 and BZ1924623) bugs were fixed.