This service will be undergoing maintenance at 00:00 UTC, 2017-10-23 It is expected to last about 30 minutes
Bug 1261094 - Jenkins builds doesn't work
Jenkins builds doesn't work
Status: CLOSED WONTFIX
Product: OpenShift Origin
Classification: Red Hat
Component: Image (Show other bugs)
2.x
Unspecified Unspecified
unspecified Severity high
: ---
: ---
Assigned To: Abhishek Gupta
DeShuai Ma
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-09-08 10:47 EDT by Dmitry Miroshnichenko
Modified: 2017-05-31 14:22 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-05-31 14:22:11 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Dmitry Miroshnichenko 2015-09-08 10:47:29 EDT
Description of problem:
Installed jenkins cartridge and jenkins client. Configured jenkins credentials(was getting 401 error). By now build starts it creates it's own builder(also creates a gear in openshift) and then fails due to those bugs(pretty close to mine):
https://bugzilla.redhat.com/show_bug.cgi?id=1128251#c5
https://issues.jenkins-ci.org/browse/JENKINS-18578

How reproducible:
Start build

Steps to Reproduce:
1.Add jekins server and client to openshift origin
2.Manage credentials
3.Run build
4.Get an error below

Actual results:
Started by user Jenkins System Builder
Building remotely on netztestbldr (netztest-build) in workspace /var/lib/openshift/55eee08346dcfd4e48000001/app-root/runtime/repo
Checkout:repo / /var/lib/openshift/55eee08346dcfd4e48000001/app-root/runtime/repo - hudson.remoting.Channel@52fc9ac8:netztestbldr
Using strategy: Default
Checkout:repo / /var/lib/openshift/55eee08346dcfd4e48000001/app-root/runtime/repo - hudson.remoting.LocalChannel@14e6ca4e
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository origin
Fetching upstream changes from ssh://55d66b0d46dcfd658c0000b2@netztest-opendingo.apps.os.netzrezepte.com/~/git/netztest.git
Seen branch in repository origin/HEAD
Seen branch in repository origin/master
Commencing build of Revision d2a9fafcda6f9c7b446280bc7e108817d1d734b0 (origin/HEAD, origin/master)
Checking out Revision d2a9fafcda6f9c7b446280bc7e108817d1d734b0 (origin/HEAD, origin/master)
java.io.IOException: remote file operation failed: /var/lib/openshift/55eee08346dcfd4e48000001/app-root/runtime/repo at hudson.remoting.Channel@52fc9ac8:netztestbldr: java.io.IOException: Remote call on netztestbldr failed
	at hudson.FilePath.act(FilePath.java:987)
	at hudson.FilePath.act(FilePath.java:969)
	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1178)
	at hudson.model.AbstractProject.checkout(AbstractProject.java:1282)
	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:610)
	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:532)
	at hudson.model.Run.execute(Run.java:1741)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:381)
Caused by: java.io.IOException: Remote call on netztestbldr failed
	at hudson.remoting.Channel.call(Channel.java:761)
	at hudson.FilePath.act(FilePath.java:980)
	... 10 more
Caused by: java.lang.Error: Unable to load resource javax/servlet/LocalStrings.properties
	at hudson.remoting.RemoteClassLoader.findResource(RemoteClassLoader.java:381)
	at java.lang.ClassLoader.getResource(ClassLoader.java:1147)
	at java.net.URLClassLoader.getResourceAsStream(URLClassLoader.java:227)
	at java.util.ResourceBundle$Control$1.run(ResourceBundle.java:2604)
	at java.util.ResourceBundle$Control$1.run(ResourceBundle.java:2589)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.util.ResourceBundle$Control.newBundle(ResourceBundle.java:2588)
	at java.util.ResourceBundle.loadBundle(ResourceBundle.java:1436)
	at java.util.ResourceBundle.findBundle(ResourceBundle.java:1400)
	at java.util.ResourceBundle.findBundle(ResourceBundle.java:1354)
	at java.util.ResourceBundle.findBundle(ResourceBundle.java:1354)
	at java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1296)
	at java.util.ResourceBundle.getBundle(ResourceBundle.java:721)
	at javax.servlet.GenericServlet.<clinit>(GenericServlet.java:95)
	at hudson.model.Node$Mode.<clinit>(Node.java:525)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at java.lang.Class.getEnumConstantsShared(Class.java:3138)
	at java.lang.Class.getEnumConstants(Class.java:3116)
	at jenkins.model.Jenkins.<clinit>(Jenkins.java:4246)
	at hudson.ExtensionList.lookup(ExtensionList.java:400)
	at hudson.FilePath.act(FilePath.java:977)
	at hudson.FilePath.act(FilePath.java:969)
	at hudson.FilePath.write(FilePath.java:1883)
	at hudson.plugins.git.GitSCM.computeChangeLog(GitSCM.java:1262)
	at hudson.plugins.git.GitSCM.access$700(GitSCM.java:79)
	at hudson.plugins.git.GitSCM$4.invoke(GitSCM.java:1233)
	at hudson.plugins.git.GitSCM$4.invoke(GitSCM.java:1178)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2691)
	at hudson.remoting.UserRequest.perform(UserRequest.java:121)
	at hudson.remoting.UserRequest.perform(UserRequest.java:49)
	at hudson.remoting.Request$2.run(Request.java:325)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at ......remote call to netztestbldr(Native Method)
	at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1361)
	at hudson.remoting.UserResponse.retrieve(UserRequest.java:221)
	at hudson.remoting.Channel.call(Channel.java:753)
	... 11 more
Caused by: java.util.concurrent.ExecutionException: java.io.IOException: Failed to write to /var/lib/openshift/55eee08346dcfd4e48000001/.jenkins/cache/jars/FD/3FEA6034F56042672E45A7A2D99DC7.jar
	at hudson.remoting.AsyncFutureImpl.get(AsyncFutureImpl.java:75)
	at hudson.remoting.FutureAdapter.get(FutureAdapter.java:55)
	at hudson.remoting.RemoteClassLoader.findResource(RemoteClassLoader.java:373)
	at java.lang.ClassLoader.getResource(ClassLoader.java:1147)
	at java.net.URLClassLoader.getResourceAsStream(URLClassLoader.java:227)
	at java.util.ResourceBundle$Control$1.run(ResourceBundle.java:2604)
	at java.util.ResourceBundle$Control$1.run(ResourceBundle.java:2589)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.util.ResourceBundle$Control.newBundle(ResourceBundle.java:2588)
	at java.util.ResourceBundle.loadBundle(ResourceBundle.java:1436)
	at java.util.ResourceBundle.findBundle(ResourceBundle.java:1400)
	at java.util.ResourceBundle.findBundle(ResourceBundle.java:1354)
	at java.util.ResourceBundle.findBundle(ResourceBundle.java:1354)
	at java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1296)
	at java.util.ResourceBundle.getBundle(ResourceBundle.java:721)
	at javax.servlet.GenericServlet.<clinit>(GenericServlet.java:95)
	at hudson.model.Node$Mode.<clinit>(Node.java:525)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at java.lang.Class.getEnumConstantsShared(Class.java:3138)
	at java.lang.Class.getEnumConstants(Class.java:3116)
	at jenkins.model.Jenkins.<clinit>(Jenkins.java:4246)
	at hudson.ExtensionList.lookup(ExtensionList.java:400)
	at hudson.FilePath.act(FilePath.java:977)
	at hudson.FilePath.act(FilePath.java:969)
	at hudson.FilePath.write(FilePath.java:1883)
	at hudson.plugins.git.GitSCM.computeChangeLog(GitSCM.java:1262)
	at hudson.plugins.git.GitSCM.access$700(GitSCM.java:79)
	at hudson.plugins.git.GitSCM$4.invoke(GitSCM.java:1233)
	at hudson.plugins.git.GitSCM$4.invoke(GitSCM.java:1178)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2691)
	at hudson.remoting.UserRequest.perform(UserRequest.java:121)
	at hudson.remoting.UserRequest.perform(UserRequest.java:49)
	at hudson.remoting.Request$2.run(Request.java:325)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Failed to write to /var/lib/openshift/55eee08346dcfd4e48000001/.jenkins/cache/jars/FD/3FEA6034F56042672E45A7A2D99DC7.jar
	at hudson.remoting.FileSystemJarCache.retrieve(FileSystemJarCache.java:85)
	at hudson.remoting.JarCacheSupport$1.run(JarCacheSupport.java:62)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at hudson.remoting.AtmostOneThreadExecutor$Worker.run(AtmostOneThreadExecutor.java:110)
	... 1 more
Caused by: java.io.IOException: No such file or directory
	at java.io.UnixFileSystem.createFileExclusively(Native Method)
	at java.io.File.createTempFile(File.java:2001)
	at hudson.remoting.FileSystemJarCache.retrieve(FileSystemJarCache.java:61)
	... 5 more
Archiving artifacts
Finished: FAILURE

Expected results:
Build passes or fails, but with another error

Additional info:
Comment 1 Dmitry Miroshnichenko 2015-09-08 10:50:58 EDT
I appreciate any solution. We need to make it work fast
Comment 2 Dmitry Miroshnichenko 2015-09-08 18:58:41 EDT
When I try to build after new builder was created and yet not destroyed - geting an error below. Maybe it will help
 
Started by user Jenkins System Builder
Building remotely on netztestbldr (netztest-build) in workspace /var/lib/openshift/55ef65b546dcfd94d5000001/app-root/runtime/repo
Checkout:repo / /var/lib/openshift/55ef65b546dcfd94d5000001/app-root/runtime/repo - hudson.remoting.Channel@4be107ff:netztestbldr
Using strategy: Default
Checkout:repo / /var/lib/openshift/55ef65b546dcfd94d5000001/app-root/runtime/repo - hudson.remoting.LocalChannel@13e13766
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository origin
Fetching upstream changes from ssh://55d66b0d46dcfd658c0000b2@netztest-opendingo.apps.os.netzrezepte.com/~/git/netztest.git
Seen branch in repository origin/HEAD
Seen branch in repository origin/master
Commencing build of Revision d2a9fafcda6f9c7b446280bc7e108817d1d734b0 (origin/HEAD, origin/master)
Checking out Revision d2a9fafcda6f9c7b446280bc7e108817d1d734b0 (origin/HEAD, origin/master)
java.io.IOException: remote file operation failed: /var/lib/openshift/55ef65b546dcfd94d5000001/app-root/runtime/repo at hudson.remoting.Channel@4be107ff:netztestbldr: java.io.IOException: Remote call on netztestbldr failed
	at hudson.FilePath.act(FilePath.java:987)
	at hudson.FilePath.act(FilePath.java:969)
	at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1178)
	at hudson.model.AbstractProject.checkout(AbstractProject.java:1282)
	at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:610)
	at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:532)
	at hudson.model.Run.execute(Run.java:1741)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:98)
	at hudson.model.Executor.run(Executor.java:381)
Caused by: java.io.IOException: Remote call on netztestbldr failed
	at hudson.remoting.Channel.call(Channel.java:761)
	at hudson.FilePath.act(FilePath.java:980)
	... 10 more
Caused by: java.lang.NoClassDefFoundError: Could not initialize class jenkins.model.Jenkins
	at hudson.ExtensionList.lookup(ExtensionList.java:400)
	at hudson.FilePath.act(FilePath.java:977)
	at hudson.FilePath.act(FilePath.java:969)
	at hudson.FilePath.write(FilePath.java:1883)
	at hudson.plugins.git.GitSCM.computeChangeLog(GitSCM.java:1262)
	at hudson.plugins.git.GitSCM.access$700(GitSCM.java:79)
	at hudson.plugins.git.GitSCM$4.invoke(GitSCM.java:1233)
	at hudson.plugins.git.GitSCM$4.invoke(GitSCM.java:1178)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2691)
	at hudson.remoting.UserRequest.perform(UserRequest.java:121)
	at hudson.remoting.UserRequest.perform(UserRequest.java:49)
	at hudson.remoting.Request$2.run(Request.java:325)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
	at ......remote call to netztestbldr(Native Method)
	at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1361)
	at hudson.remoting.UserResponse.retrieve(UserRequest.java:221)
	at hudson.remoting.Channel.call(Channel.java:753)
	... 11 more
Archiving artifacts
Finished: FAILURE
Comment 3 Dmitry Miroshnichenko 2015-09-17 15:57:32 EDT
Finally, found the cause. System acually tries to write directory, which doesn't exist. I don't know how, but after some days of googling have found this source file:
https://github.com/openshift/jenkins-cloud-plugin/blob/master/src/main/java/hudson/plugins/openshift/OpenShiftComputerLauncher.java#L110

We see that there is a strange JENKINS_JAR_CACHE_PATH environment variable. Actually there was nothing about such variable that should be created by Openshift. 

Some time later have found proper recipe, thanks Mateus for sharing:
https://lists.openshift.redhat.com/openshift-archives/dev/2015-June/msg00027.html
2015-06-13 0:21 GMT+03:00 Mateus Caruccio <mateus caruccio getupcloud com>:
Just for the record, my problem started when I manually updated my jenkins server to 1.510, and this bug came out: https://issues.jenkins-ci.org/browse/JENKINS-18578 - slave.jar tries to place jar cache under ~/.jenkins/cache/jars, which is forbidden to write.

Workaround:
# touch /etc/openshift/env/JENKINS_JAR_CACHE_PATH
# cat <<EOF > /etc/openshift/env/JENKINS_JAR_CACHE_PATH
\$OPENSHIFT_DATA_DIR/.jenkins/cache/jars
EOF
# reboot

Result:
Builds run as they should

Additional links (might be usefull):
http://stackoverflow.com/questions/24083455/openshift-jenkins-does-not-appear-to-send-node-label-during-build
Comment 4 Eric Paris 2017-05-31 14:22:11 EDT
We apologize, however, we do not plan to address this report at this time. The majority of our active development is for the v3 version of OpenShift. If you would like for Red Hat to reconsider this decision, please reach out to your support representative. We are very sorry for any inconvenience this may cause.

Note You need to log in before you can comment on or make changes to this bug.