Description of problem: Setup multi-node env and create district on it, add two node to the district. Create jenkins server in the district. After move the jenkins server from one node to the other, build app with jenkins will always fail. For both existed app with jenkins added before move and new created app after move. Version-Release number of selected component (if applicable): devenv_4027 How reproducible: always Steps to Reproduce: 1.Setup multi-node env with district on it 2.Create jenkins server and create app with jenkins added 3.Move the jenkins server to other node #oo-admin-move --gear_uuid 5282eb7f51b0a03198000001 -i ip-10-151-71-11 4.Try to build the existed app with jenkins 5.Create new app with jenkins added and build Actual results: Jenkins build will always fail for step4 and step5. Expected results: Should be able to build with jenkins after move. Additional info: The broker_host has been set to the broker IP in node.conf. PUBLIC_HOSTNAME=ec2-107-22-83-54.compute-1.amazonaws.com PUBLIC_IP=107.22.83.54 BROKER_HOST=10.151.120.101 Jenkins log for the failure. Nov 13, 2013 4:51:47 AM hudson.plugins.openshift.OpenShiftCloud provision INFO: Provisioning new node for workload = 2 and label = app5-build Nov 13, 2013 4:51:47 AM hudson.plugins.openshift.OpenShiftCloud getOpenShiftConnection INFO: Initiating Java Client Service - Configured for OpenShift Server https://localhost Nov 13, 2013 4:51:47 AM com.openshift.internal.client.RestService request INFO: Requesting GET with protocol 1.2 on https://localhost/broker/rest/api Nov 13, 2013 4:51:47 AM com.openshift.internal.client.RestService request INFO: Requesting GET with protocol 1.2 on https://localhost/broker/rest/user Nov 13, 2013 4:51:47 AM hudson.plugins.openshift.OpenShiftCloud provision WARNING: Caught com.openshift.client.NotFoundOpenShiftException: Could not find any OpenShift resource at "https://localhost/broker/rest/user". Will retry 4 more times before canceling build. Nov 13, 2013 4:51:52 AM com.openshift.internal.client.RestService request INFO: Requesting GET with protocol 1.2 on https://localhost/broker/rest/user Nov 13, 2013 4:51:52 AM hudson.plugins.openshift.OpenShiftCloud provision WARNING: Caught com.openshift.client.NotFoundOpenShiftException: Could not find any OpenShift resource at "https://localhost/broker/rest/user". Will retry 3 more times before canceling build. Nov 13, 2013 4:51:57 AM com.openshift.internal.client.RestService request INFO: Requesting GET with protocol 1.2 on https://localhost/broker/rest/user Nov 13, 2013 4:51:57 AM hudson.plugins.openshift.OpenShiftCloud provision WARNING: Caught com.openshift.client.NotFoundOpenShiftException: Could not find any OpenShift resource at "https://localhost/broker/rest/user". Will retry 2 more times before canceling build. Nov 13, 2013 4:52:02 AM com.openshift.internal.client.RestService request INFO: Requesting GET with protocol 1.2 on https://localhost/broker/rest/user Nov 13, 2013 4:52:02 AM hudson.plugins.openshift.OpenShiftCloud provision WARNING: Caught com.openshift.client.NotFoundOpenShiftException: Could not find any OpenShift resource at "https://localhost/broker/rest/user". Will retry 1 more times before canceling build. Nov 13, 2013 4:52:07 AM com.openshift.internal.client.RestService request INFO: Requesting GET with protocol 1.2 on https://localhost/broker/rest/user Nov 13, 2013 4:52:07 AM hudson.plugins.openshift.OpenShiftCloud provision WARNING: Caught com.openshift.client.NotFoundOpenShiftException: Could not find any OpenShift resource at "https://localhost/broker/rest/user". Will retry 0 more times before canceling build. Nov 13, 2013 4:52:12 AM hudson.plugins.openshift.OpenShiftCloud provision WARNING: Cancelling build due to earlier exceptions com.openshift.client.NotFoundOpenShiftException: Could not find any OpenShift resource at "https://localhost/broker/rest/user" at com.openshift.internal.client.RestService.request(RestService.java:132) at com.openshift.internal.client.RestService.request(RestService.java:109) at com.openshift.internal.client.RestService.request(RestService.java:92) at com.openshift.internal.client.AbstractOpenShiftResource$ServiceRequest.execute(AbstractOpenShiftResource.java:139) at com.openshift.internal.client.APIResource$GetUserRequest.execute(APIResource.java:245) at com.openshift.internal.client.APIResource.getUser(APIResource.java:100) at hudson.plugins.openshift.OpenShiftCloud.getSlaves(OpenShiftCloud.java:863) at hudson.plugins.openshift.OpenShiftCloud.provisionSlave(OpenShiftCloud.java:440) at hudson.plugins.openshift.OpenShiftCloud.provision(OpenShiftCloud.java:402) at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:264) at hudson.slaves.NodeProvisioner.access$000(NodeProvisioner.java:51) at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:347) at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54) at java.util.TimerThread.mainLoop(Timer.java:555) at java.util.TimerThread.run(Timer.java:505) Caused by: com.openshift.internal.client.httpclient.NotFoundException: {"api_version":1.2,"data":null,"messages":[{"exit_code":101,"field":null,"index":null,"severity":"error","text":"Application '52834a6a51b0a0d7f1000339' not found."}],"status":"not_found","supported_api_versions":[1.0,1.1,1.2,1.3,1.4,1.5,1.6],"type":null,"version":"1.2"} at com.openshift.internal.client.httpclient.UrlConnectionHttpClient.createException(UrlConnectionHttpClient.java:159) at com.openshift.internal.client.httpclient.UrlConnectionHttpClient.request(UrlConnectionHttpClient.java:134) at com.openshift.internal.client.httpclient.UrlConnectionHttpClient.request(UrlConnectionHttpClient.java:114) at com.openshift.internal.client.httpclient.UrlConnectionHttpClient.get(UrlConnectionHttpClient.java:84) at com.openshift.internal.client.RestService.request(RestService.java:175) at com.openshift.internal.client.RestService.request(RestService.java:123) ... 14 more Caused by: java.io.FileNotFoundException: https://localhost/broker/rest/user at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1624) at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254) at com.openshift.internal.client.httpclient.UrlConnectionHttpClient.request(UrlConnectionHttpClient.java:130) ... 18 more Nov 13, 2013 4:52:12 AM hudson.plugins.openshift.OpenShiftCloud cancelItem INFO: Cancelling Item Nov 13, 2013 4:52:12 AM hudson.plugins.openshift.OpenShiftCloud cancelItem WARNING: Build app5-build app5bldr has been canceled
The issue here is the java client looks for /etc/openshift/express.conf for it's preferred settings. If you move that file or change localhost in it the problem will not occur. I have fixed the devenv steps to create a multinode devenv with this step: https://github.com/openshift/li/pull/2148
Commit pushed to master at https://github.com/openshift/li https://github.com/openshift/li/commit/e9044bb67e94065dda9e07d9b9c6a4a129545a66 Bug 1029828
Tried with the latest devenv script. The jenkins build will be success after moved the jenkins server. Nov 14, 2013 1:06:24 AM hudson.plugins.openshift.OpenShiftCloud getOpenShiftConnection INFO: Initiating Java Client Service - Configured for OpenShift Server https://10.147.162.122 Nov 14, 2013 1:06:24 AM com.openshift.internal.client.RestService request INFO: Requesting GET with protocol 1.2 on https://10.147.162.122/broker/rest/api Nov 14, 2013 1:06:25 AM com.openshift.internal.client.RestService request INFO: Requesting GET with protocol 1.2 on https://10.147.162.122/broker/rest/user Nov 14, 2013 1:06:25 AM com.openshift.internal.client.RestService request INFO: Requesting GET with protocol 1.2 on https://10.147.162.122/broker/rest/domains Nov 14, 2013 1:06:25 AM com.openshift.internal.client.RestService request INFO: Requesting GET with protocol 1.2 on https://10.147.162.122/broker/rest/domain/bmengdev/applications?include=cartridges Nov 14, 2013 1:06:25 AM hudson.plugins.openshift.OpenShiftCloud hasCapacity INFO: Checking capacity Nov 14, 2013 1:06:25 AM hudson.plugins.openshift.OpenShiftCloud hasCapacity INFO: User has consumed 2 of 50 gears. Move bug to verified.