Bug 971789 - [upgrade] jenkins build does not work after migrate.
[upgrade] jenkins build does not work after migrate.
Product: OpenShift Container Platform
Classification: Red Hat
Component: Upgrade (Show other bugs)
Unspecified Unspecified
medium Severity medium
: ---
: ---
Assigned To: chris alfonso
libra bugs
Depends On: 971786
  Show dependency treegraph
Reported: 2013-06-07 05:38 EDT by Johnny Liu
Modified: 2017-03-08 12 EST (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2013-06-27 11:24:12 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Johnny Liu 2013-06-07 05:38:52 EDT
Description of problem:
jenkins build does not work after migrate.

$ git commit -a -mx; git push
Writing objects: 100% (4/4), 366 bytes, done.
Total 4 (delta 2), reused 0 (delta 0)
remote: Executing Jenkins build.
remote: You can track your build at https://jenkins-jialiu.ose11test.com/job/pythonscal-build
remote: Waiting for build to schedule.............................
remote: Please see the Jenkins log for more details via 'rhc tail'
remote: !!!!!!!!
remote: Deployment Halted!
remote: If the build failed before the deploy step, your previous
remote: build is still running.  Otherwise, your application may be
remote: partially deployed or inaccessible.
remote: Fix the build and try again.
remote: !!!!!!!!
remote: An error occurred executing 'gear postreceive'
remote: For more details about the problem, try running the command again with the '--trace' option.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Setup ose-1.1.3 env
2. Create a jenkins app, and create another app with jenkins-client embedded, make sure jenkins build is working fine.
3. Following http://etherpad.corp.redhat.com/OSE-1-2-upgrade-notes to do upgrade testing
4. After upgrade, do some change in app repo to trigger jenkins build.

Actual results:
Fail to do jenkins build.

Check jenkins server log, found that:
INFO: Requesting GET on https://broker.ose11test.com/broker/rest/user
Jun 07, 2013 1:56:21 AM hudson.plugins.openshift.OpenShiftCloud provision
WARNING: Caught com.openshift.client.InvalidCredentialsOpenShiftException: Your credentials are not authorized to access "https://broker.ose11test.com/broker/rest/user". Will retry 4 more times before canceling build.
com.openshift.client.InvalidCredentialsOpenShiftException: Your credentials are not authorized to access "https://broker.ose11test.com/broker/rest/user"
	at com.openshift.internal.client.RestService.request(RestService.java:102)
	at com.openshift.internal.client.RestService.request(RestService.java:91)
	at com.openshift.internal.client.RestService.request(RestService.java:76)
	at com.openshift.internal.client.AbstractOpenShiftResource$ServiceRequest.execute(AbstractOpenShiftResource.java:124)
	at com.openshift.internal.client.APIResource$GetUserRequest.execute(APIResource.java:239)
	at com.openshift.internal.client.APIResource.getUser(APIResource.java:98)
	at hudson.plugins.openshift.OpenShiftCloud.getSlaves(OpenShiftCloud.java:859)
	at hudson.plugins.openshift.OpenShiftCloud.provisionSlave(OpenShiftCloud.java:436)
	at hudson.plugins.openshift.OpenShiftCloud.provision(OpenShiftCloud.java:401)
	at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:264)
	at hudson.slaves.NodeProvisioner.access$000(NodeProvisioner.java:51)
	at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:347)
	at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54)
	at java.util.TimerThread.mainLoop(Timer.java:555)
	at java.util.TimerThread.run(Timer.java:505)
Caused by: com.openshift.internal.client.httpclient.UnauthorizedException: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<title>401 Authorization Required</title>
<h1>Authorization Required</h1>
<p>This server could not verify that you
are authorized to access the document
requested.  Either you supplied the wrong
credentials (e.g., bad password), or your
browser doesn't understand how to supply
the credentials required.</p>
<address>Apache/2.2.15 (Red Hat) Server at broker.ose11test.com Port 80</address>

Expected results:
Jenkins build should succeed.

Additional info:
Comment 2 Luke Meyer 2013-06-07 11:22:05 EDT
Refer to https://www.openshift.com/blogs/important-version-two-cartidge-information-for-jenkins-users
 - it looks like this migration may break existing Jenkins clients.

Can you try removing and re-adding the jenkins-client cartridge to see if that fixes things?

If this is the problem, we may just have to document it as a known issue in the release notes. I don't think there's an easy way to automate this.
Comment 3 Johnny Liu 2013-06-08 09:11:30 EDT
No, following the document, still does NOT work, the same error is seen in jenkins log.
Comment 4 Brenton Leanhardt 2013-06-11 16:05:19 EDT
I can definitely see something odd is happening inside the java client.  I have verified with curl that the authkey and authtoken are not being sent to the Broker (thus the 401).

I setup a new 1.2 install and jenkins obviously worked as expected.  The version of the jenkins plugin and the java client are the same.  Somehow a different code path is being triggered since the requests to /broker/rest/user have a different user agents.  On the broken setup the user agent is 'User-Agent: Java/1.7.0_19' (the version of the java client).  On a working setup the user agent is 'OpenShift'.  The later is required for authentication to passthrough properly.
Comment 8 John W. Lamb 2013-06-21 15:44:11 EDT
Bouncing over to calfonso for workaround documentation
Comment 9 Luke Meyer 2013-06-27 11:24:12 EDT
We have a generic re-create-everything note in the upgrade notes. If anything more detailed needs to be described, we can create a kbase article and link it from the release notes.
Comment 10 Johnny Liu 2013-07-01 07:52:37 EDT
Today I re-test this bug using RC2 build, found https://www.openshift.com/blogs/important-version-two-cartidge-information-for-jenkins-users could address this issue.

Note You need to log in before you can comment on or make changes to this bug.