Bug 1813217

Summary: Must add ip address wildcard to no_proxy for jenkins
Product: OpenShift Container Platform Reporter: XiuJuan Wang <xiuwang>
Component: JenkinsAssignee: jawed <jkhelil>
Status: CLOSED ERRATA QA Contact: Jitendar Singh <jitsingh>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.4CC: abenaiss, aos-bugs, pbhattac, vbobade
Target Milestone: ---   
Target Release: 4.6.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-10-27 15:56:55 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description XiuJuan Wang 2020-03-13 10:12:10 UTC
Description of problem:
When pass proxy settting to jenkins dc via env vars under proxy cluster, pipeline build still failed.
```
oc set env dc/jenkins http_proxy=$(oc get proxy cluster -o jsonpath='{.status.httpProxy}') \
                      https_proxy=$(oc get proxy cluster -o jsonpath='{.status.httpsProxy}') \
                      no_proxy=$(oc get proxy cluster -o jsonpath='{.status.noProxy}') 
```
Must add ip wildcard equivalent to no_proxy for jenkins 

Version-Release number of selected component (if applicable):

4.4.0-0.nightly-2020-03-12-052849 
How reproducible:

always

Steps to Reproduce:
1.Install a cluster and enabled proxy, create a jenkins application
2. set proxy for jenkins
3.Trigger a pipeline build

Actual results:
pipeline build failed.

Expected results:

Must add ip address wildcard to no_proxy for jenkins, pipeline build should succeed.

Additional info:
oc logs -f dc/jenkins
2020-03-13 09:19:44.491+0000 [id=33]	INFO	i.f.j.o.BuildSyncRunListener#upsertBuild: skipping stage Initialize for the status JSON for pipeline run #1 because it was not executed (most likely because of a failure in another stage)
2020-03-13 09:19:49.491+0000 [id=36]	INFO	i.f.j.o.BuildSyncRunListener#upsertBuild: skipping stage Initialize for the status JSON for pipeline run #1 because it was not executed (most likely because of a failure in another stage)
2020-03-13 09:19:52.249+0000 [id=33]	INFO	o.c.j.p.k.KubernetesCloud#provision: Excess workload after pending Kubernetes agents: 1
2020-03-13 09:19:52.249+0000 [id=33]	INFO	o.c.j.p.k.KubernetesCloud#provision: Template for label maven: Kubernetes Pod Template
2020-03-13 09:19:52.252+0000 [id=33]	WARNING	o.c.j.p.k.KubernetesCloud#provision: Failed to count the # of live instances on Kubernetes
java.io.IOException: Failed to authenticate with proxy
	at okhttp3.internal.connection.RealConnection.createTunnel(RealConnection.java:402)
	at okhttp3.internal.connection.RealConnection.connectTunnel(RealConnection.java:219)
	at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:160)
	at okhttp3.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:257)
	at okhttp3.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:135)
	at okhttp3.internal.connection.StreamAllocation.newStream(StreamAllocation.java:114)
	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:126)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
	at io.fabric8.kubernetes.client.utils.BackwardsCompatibilityInterceptor.intercept(BackwardsCompatibilityInterceptor.java:119)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
	at io.fabric8.kubernetes.client.utils.ImpersonatorInterceptor.intercept(ImpersonatorInterceptor.java:68)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
	at io.fabric8.kubernetes.client.utils.HttpClientUtils.lambda$createHttpClient$3(HttpClientUtils.java:110)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:254)
	at okhttp3.RealCall.execute(RealCall.java:92)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:404)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:365)
	at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:347)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.listRequestHelper(BaseOperation.java:145)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:612)
	at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:63)
	at org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud.getActiveSlavePods(KubernetesCloud.java:581)
	at org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud.addProvisionedSlave(KubernetesCloud.java:556)
	at org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud.provision(KubernetesCloud.java:508)
	at hudson.slaves.NodeProvisioner$StandardStrategyImpl.apply(NodeProvisioner.java:725)
	at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:332)
	at hudson.slaves.NodeProvisioner.access$900(NodeProvisioner.java:63)
	at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:819)
	at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:70)
	at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Comment 3 Akram Ben Aissi 2020-06-03 12:58:11 UTC
not a blocker for 4.5
we will do it in z stream

Comment 7 XiuJuan Wang 2020-06-10 07:07:43 UTC
@Jawed @Akram
While the fix merged, we no need the workaround to add proxy env to jenkins pod as https://bugzilla.redhat.com/show_bug.cgi?id=1780125#c4 , then we could access jenkins webconsole.
But pipelibe build still can't access github as https://bugzilla.redhat.com/show_bug.cgi?id=1753562#c0 .

I will mark this bug as verified in 4.6.0-0.nightly-2020-06-09-234748, using https://bugzilla.redhat.com/show_bug.cgi?id=1753562 to track the pipeline build on proxy issue.

Comment 8 jawed 2020-06-10 07:16:28 UTC
@xiuwang
the fix here, fixes the pipelinebuild with the maven agent when a proxy is set.
but it wont work until this one is merged, because, that proxy settings needs to be fixed on the jenkins master also here is the PR https://github.com/openshift/jenkins/pull/1078(which is a fix for https://bugzilla.redhat.com/show_bug.cgi?id=1780125#c4) please take a look to my last comment on it, to make it work, the no_proxy env should be set correctly.
So basically to resume this: I fixed the proxy handling in jenkins and on maven agent. but still this need to pass correct no_proxy env variable.
This one https://bugzilla.redhat.com/show_bug.cgi?id=1753562#c0 , as I put on my last comment is not related to jenkins directly, it should be fixed on openshift component. but it is not mandatory, because it just set the proxy env variables globaly, unforthenately it is not working, that's why we set them via oc set env command.

Comment 10 errata-xmlrpc 2020-10-27 15:56:55 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196