Red Hat Bugzilla – Bug 971789
[upgrade] jenkins build does not work after migrate.
Last modified: 2017-03-08 12:35 EST
Description of problem: jenkins build does not work after migrate. $ git commit -a -mx; git push <--snip--> Writing objects: 100% (4/4), 366 bytes, done. Total 4 (delta 2), reused 0 (delta 0) remote: Executing Jenkins build. remote: remote: You can track your build at https://jenkins-jialiu.ose11test.com/job/pythonscal-build remote: remote: Waiting for build to schedule............................. remote: **BUILD FAILED/CANCELLED** remote: Please see the Jenkins log for more details via 'rhc tail' remote: !!!!!!!! remote: Deployment Halted! remote: If the build failed before the deploy step, your previous remote: build is still running. Otherwise, your application may be remote: partially deployed or inaccessible. remote: Fix the build and try again. remote: !!!!!!!! remote: An error occurred executing 'gear postreceive' remote: remote: For more details about the problem, try running the command again with the '--trace' option. <--snip--> Version-Release number of selected component (if applicable): 1.2/2013-06-05.9 How reproducible: Always Steps to Reproduce: 1. Setup ose-1.1.3 env 2. Create a jenkins app, and create another app with jenkins-client embedded, make sure jenkins build is working fine. 3. Following http://etherpad.corp.redhat.com/OSE-1-2-upgrade-notes to do upgrade testing 4. After upgrade, do some change in app repo to trigger jenkins build. Actual results: Fail to do jenkins build. Check jenkins server log, found that: INFO: Requesting GET on https://broker.ose11test.com/broker/rest/user Jun 07, 2013 1:56:21 AM hudson.plugins.openshift.OpenShiftCloud provision WARNING: Caught com.openshift.client.InvalidCredentialsOpenShiftException: Your credentials are not authorized to access "https://broker.ose11test.com/broker/rest/user". Will retry 4 more times before canceling build. com.openshift.client.InvalidCredentialsOpenShiftException: Your credentials are not authorized to access "https://broker.ose11test.com/broker/rest/user" at com.openshift.internal.client.RestService.request(RestService.java:102) at com.openshift.internal.client.RestService.request(RestService.java:91) at com.openshift.internal.client.RestService.request(RestService.java:76) at com.openshift.internal.client.AbstractOpenShiftResource$ServiceRequest.execute(AbstractOpenShiftResource.java:124) at com.openshift.internal.client.APIResource$GetUserRequest.execute(APIResource.java:239) at com.openshift.internal.client.APIResource.getUser(APIResource.java:98) at hudson.plugins.openshift.OpenShiftCloud.getSlaves(OpenShiftCloud.java:859) at hudson.plugins.openshift.OpenShiftCloud.provisionSlave(OpenShiftCloud.java:436) at hudson.plugins.openshift.OpenShiftCloud.provision(OpenShiftCloud.java:401) at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:264) at hudson.slaves.NodeProvisioner.access$000(NodeProvisioner.java:51) at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:347) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54) at java.util.TimerThread.mainLoop(Timer.java:555) at java.util.TimerThread.run(Timer.java:505) Caused by: com.openshift.internal.client.httpclient.UnauthorizedException: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>401 Authorization Required</title> </head><body> <h1>Authorization Required</h1> <p>This server could not verify that you are authorized to access the document requested. Either you supplied the wrong credentials (e.g., bad password), or your browser doesn't understand how to supply the credentials required.</p> <hr> <address>Apache/2.2.15 (Red Hat) Server at broker.ose11test.com Port 80</address> </body></html> Expected results: Jenkins build should succeed. Additional info:
Refer to https://www.openshift.com/blogs/important-version-two-cartidge-information-for-jenkins-users - it looks like this migration may break existing Jenkins clients. Can you try removing and re-adding the jenkins-client cartridge to see if that fixes things? If this is the problem, we may just have to document it as a known issue in the release notes. I don't think there's an easy way to automate this.
No, following the document, still does NOT work, the same error is seen in jenkins log.
I can definitely see something odd is happening inside the java client. I have verified with curl that the authkey and authtoken are not being sent to the Broker (thus the 401). I setup a new 1.2 install and jenkins obviously worked as expected. The version of the jenkins plugin and the java client are the same. Somehow a different code path is being triggered since the requests to /broker/rest/user have a different user agents. On the broken setup the user agent is 'User-Agent: Java/1.7.0_19' (the version of the java client). On a working setup the user agent is 'OpenShift'. The later is required for authentication to passthrough properly.
Bouncing over to calfonso for workaround documentation
We have a generic re-create-everything note in the upgrade notes. If anything more detailed needs to be described, we can create a kbase article and link it from the release notes.
Today I re-test this bug using RC2 build, found https://www.openshift.com/blogs/important-version-two-cartidge-information-for-jenkins-users could address this issue.