Not sure yet what could have triggered it ... could be - the recent bump to jenkins 2.190.3 via https://github.com/openshift/jenkins/pull/978 though CI passed there - or an underlying change in OpenShift somewhere I think in the Dec 12th time frame But jenkins /openshift oauth integration is now broken, where the following error happens on start up: 2019-12-13 21:04:11.252+0000 [id=18] INFO o.o.j.p.o.OpenShiftOAuth2SecurityRealm#populateDefaults: populateDefaults javax.net.ssl.SSLHandshakeException: extension (5) should not be presented in certificate_request at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131) at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:307) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:263) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:254) at java.base/sun.security.ssl.SSLExtensions.<init>(SSLExtensions.java:90) at java.base/sun.security.ssl.CertificateRequest$T13CertificateRequestMessage.<init>(CertificateRequest.java:818) at java.base/sun.security.ssl.CertificateRequest$T13CertificateRequestConsumer.consume(CertificateRequest.java:922) at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392) at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443) at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:421) at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:177) at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:164) at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1152) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1063) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:402) at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:567) at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185) at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:168) at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:143) at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:79) at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:996) at org.openshift.jenkins.plugins.openshiftlogin.OpenShiftOAuth2SecurityRealm.getOpenShiftUserInfo(OpenShiftOAuth2SecurityRealm.java:655) at org.openshift.jenkins.plugins.openshiftlogin.OpenShiftOAuth2SecurityRealm.populateDefaults(OpenShiftOAuth2SecurityRealm.java:378) at org.openshift.jenkins.plugins.openshiftlogin.OpenShiftSetOAuth.setOauth(OpenShiftSetOAuth.java:79) at org.openshift.jenkins.plugins.openshiftlogin.OpenShiftPermissionFilter.doFilter(OpenShiftPermissionFilter.java:126) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:243) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at io.jenkins.blueocean.ResourceCacheControl.doFilter(ResourceCacheControl.java:134) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at io.jenkins.blueocean.auth.jwt.impl.JwtAuthenticationFilter.doFilter(JwtAuthenticationFilter.java:61) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at jenkins.metrics.impl.MetricsFilter.doFilter(MetricsFilter.java:125) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at jenkins.telemetry.impl.UserLanguages$AcceptLanguageFilter.doFilter(UserLanguages.java:128) at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151) at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:157) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:64) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84) at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:135) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249) at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67) at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87) at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90) at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:505) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804) at java.base/java.lang.Thread.run(Thread.java:834) I am looking at the recent change logs in the 4.4 nightlies to see if I can narrow down if the change is there. But I need Akram / Vibhav to do some regression testing on post Dec 13 clusters to see if bumping the jenkins version back down has any bearing, and/or adding some temporary debug in the certificate establishment paths to see if any more light can be shed on what is going on.
Update from @gmontero: Recent Bump to Jenkins 2.190.3 is not the issue
This related to this: https://github.com/golang/go/issues/35722 As the bug has been reported to golang I doubt that openjdk team is aware of it.
This is blocked by https://bugzilla.redhat.com/show_bug.cgi?id=1784147
Reducing to high severity as we do have a way to keep this working (using jdk1.8) and we are trying to avoid deferring anything that is truly "urgent". From what I can tell progress on this is still blocked by https://bugzilla.redhat.com/show_bug.cgi?id=1784147
Related to https://bugs.openjdk.java.net/browse/JDK-8236039 which is been resolved
@Akram - isn't there a QA person for openshift/jenkins that got assigned over the last few months? If so, we should probably change the QA contact on this bug from @XiuJuan to that person. thanks
$java -version openjdk version "11.0.7" 2020-04-14 LTS OpenJDK Runtime Environment 18.9 (build 11.0.7+10-LTS) OpenJDK 64-Bit Server VM 18.9 (build 11.0.7+10-LTS, mixed mode, sharing) Could access jenkins webconsole with google oauth and htpasswd oauth after set JDK 11 as default. Verified with 4.5.0-0.nightly-2020-05-04-113741 payload.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:2409