Bug 1809924

Summary:	Can't access jenkins webconsole with 403 Forbidden
Product:	OpenShift Container Platform	Reporter:	XiuJuan Wang <xiuwang>
Component:	Jenkins	Assignee:	Akram Ben Aissi <abenaiss>
Status:	CLOSED ERRATA	QA Contact:	XiuJuan Wang <xiuwang>
Severity:	high	Docs Contact:
Priority:	high
Version:	4.3.z	CC:	aos-bugs, bparees, gmontero, jdelft, jforrest, lmohanty, pbhattac, sponnaga, vbobade, vlaad
Target Milestone:	---	Keywords:	Regression
Target Release:	4.3.z
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1810389 (view as bug list)		Environment:
Last Closed:	2020-03-10 23:54:20 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1809923
Bug Blocks:	1802297, 1810389

Description XiuJuan Wang 2020-03-04 08:24:06 UTC

This bug was initially created as a copy of Bug #1809923

I am copying this bug because: 

Description of problem:
Can't access jenkins webconsole with 403 Forbidden

Version-Release number of selected component (if applicable):
4.3.0-0.nightly-2020-03-02-115939

How reproducible:
always

Steps to Reproduce:
1.Login cluster with a common user(with htpasswd auth),Create a jenkins application
$oc new-app jenkins-persistent
2.Access jenkins webconsole after pod is ready
3.

Actual results:

Step 2: Failed to access the jenkins webconsole.
com.google.api.client.auth.oauth2.TokenResponseException: 403 Forbidden
{
  "kind" : "Status",
  "apiVersion" : "v1",
  "metadata" : { },
  "status" : "Failure",
  "message" : "forbidden: User \"system:anonymous\" cannot post path \"/oauth/token\"",
  "reason" : "Forbidden",
  "details" : { },
  "code" : 403
}
	at com.google.api.client.auth.oauth2.TokenResponseException.from(TokenResponseException.java:105)
	at com.google.api.client.auth.oauth2.TokenRequest.executeUnparsed(TokenRequest.java:326)
	at com.google.api.client.auth.openidconnect.IdTokenResponse.execute(IdTokenResponse.java:120)
	at org.openshift.jenkins.plugins.openshiftlogin.BearerTokenOAuthSession.onSuccess(BearerTokenOAuthSession.java:62)
	at org.openshift.jenkins.plugins.openshiftlogin.OAuthSession.doFinishLogin(OAuthSession.java:132)
	at org.openshift.jenkins.plugins.openshiftlogin.OpenShiftOAuth2SecurityRealm.doFinishLogin(OpenShiftOAuth2SecurityRealm.java:1095)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627)
	at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:396)
	at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:408)
	at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:212)
	at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:145)
	at org.kohsuke.stapler.MetaClass$11.doDispatch(MetaClass.java:535)
	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
	at org.kohsuke.stapler.MetaClass$2.doDispatch(MetaClass.java:219)
	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:676)
	at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:873)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1623)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
	at org.openshift.jenkins.plugins.openshiftlogin.OpenShiftPermissionFilter.doFilter(OpenShiftPermissionFilter.java:236)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:243)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at jenkins.security.ResourceDomainFilter.doFilter(ResourceDomainFilter.java:76)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at io.jenkins.blueocean.ResourceCacheControl.doFilter(ResourceCacheControl.java:134)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at io.jenkins.blueocean.auth.jwt.impl.JwtAuthenticationFilter.doFilter(JwtAuthenticationFilter.java:61)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at jenkins.metrics.impl.MetricsFilter.doFilter(MetricsFilter.java:125)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at jenkins.telemetry.impl.UserLanguages$AcceptLanguageFilter.doFilter(UserLanguages.java:128)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:157)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:64)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
	at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:135)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
	at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
	at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.Server.handle(Server.java:505)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804)
	at java.lang.Thread.run(Thread.java:748)

Expected results:
Should access jenkins successfully

Additional info:
Could access jenkins webconsole in 4.3.3 cluster

Comment 2 Akram Ben Aissi 2020-03-04 23:16:15 UTC

The oauth token endpoint between 3.x and 4.3 has changed. I have identified that the method that we use to determine if an OCP server is 3.x or 4.x is buggy.
I am testing a fix now and see if it solves the issue.

Comment 3 Akram Ben Aissi 2020-03-05 07:11:40 UTC

The fix of the jenkins plugin related to this is in this PR: https://github.com/openshift/jenkins-openshift-login-plugin/pull/99

Now starting the process for releasing the plugin.

Comment 20 XiuJuan Wang 2020-03-06 02:45:48 UTC

@Akram @Gabe
Verify the jenkins in 4.3.0-0.nightly-2020-03-06-003558 payload.
With openshift-login-plugin 1.0.23 installed by default, jenkins webconsole could be accessed with htpasswd/google auth successfully.
Thank you for all efforts ,so fast and correct response!

Comment 22 Ben Parees 2020-03-06 18:41:06 UTC

moving back to POST so we can re-verify this bug after https://github.com/openshift/jenkins/pull/1030 merges.

Comment 24 errata-xmlrpc 2020-03-10 23:54:20 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0676