Bug 797775

Summary: Existing jenkins server could not launch slave to build jobs after upgraded
Product: OKD Reporter: Xiaoli Tian <xtian>
Component: ContainersAssignee: Dan McPherson <dmcphers>
Status: CLOSED NOTABUG QA Contact: libra bugs <libra-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 1.xCC: rmillner
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-02-28 16:29:55 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
jenkins error log
none
Part of development log
none
migration logs
none
jenkins server logs none

Description Xiaoli Tian 2012-02-27 09:41:52 UTC
Created attachment 565995 [details]
jenkins error  log

Description of problem:
After updating the instance to the latest and running the migrate scripts, the existing jenkins server on the old instance can not launch slave,that means can not build job with the jenkins server,some of log is as following:

WARNING: Provisioned slave appjbosswjenkins-build failed to launch
java.io.IOException: Failed to connect to application appjbosswjenkinsbldr


Version-Release number of selected component (if applicable):
devenv_1642
migrate-2.0.6
rhc-cartridge-jenkins-1.4-0.87.3-1.el6_2.noarch
jenkins-plugin-openshift-0.4.22-1.el6_2.x86_64

How reproducible:
Always

Steps to Reproduce:
1.Create an app with jenkins embedded in devenv-stage_136
2.update and migrate on devenv-stage_136
3.Try to make some changes and trigger build by git push 
  
Actual results:
Can not launch a slave to build

Expected results:
Could build successfully

Additional info:
See No node available error in development.log
Please find other detail logs in attachments

Comment 1 Xiaoli Tian 2012-02-27 09:43:49 UTC
Created attachment 565996 [details]
Part of development log

Comment 2 Xiaoli Tian 2012-02-27 09:56:44 UTC
Created attachment 566000 [details]
migration logs

Comment 3 Dan McPherson 2012-02-27 15:48:33 UTC
Is this not the issue of not having a large instance available?

Comment 4 Xiaoli Tian 2012-02-28 01:29:49 UTC
(In reply to comment #3)
> Is this not the issue of not having a large instance available?

Hi,Dan

The environment I used was just a single node (a large instance),and had modified the resource to large as well.

ls -lh resource_limits.conf
lrwxrwxrwx. 1 root root 26 Feb 26 22:22 resource_limits.conf -> resource_limits.conf.large

I'll try to confirm if it's an issue again today.

Thanks

Comment 5 Xiaoli Tian 2012-02-28 09:42:23 UTC
I have recreated this issue today, I embed jenkins client into a ruby app to make sure there's no regression caused by jboss.

Recreate steps:

1. Launch an devenv-stage_136 instance

2. Create an ruby-1.8 app with jenkins enabled

3. Make some changes to ruby app and git push to trigger jenkins build to make sure it's ok before upgrade

4. Edit yum repo to the dev repo 

5.update ,migrate-mongo-2.0.6 and migrate-2.0.6

6.service libra restart

7.make some changes to your ruby app and git push again

Then it will keep waiting for build to schedule,jenkins will not find available executor,see jenkins log below in the attachment for details.

Comment 6 Xiaoli Tian 2012-02-28 09:43:05 UTC
Created attachment 566270 [details]
jenkins server logs

Comment 7 Dan McPherson 2012-02-28 16:29:55 UTC
I wasn't able to recreate still so I logged into your instance and noticed:

[root@ip-10-12-78-238 ~]# mc-facts node_profile
Report for fact: node_profile

        large                                   found 1 times

[root@ip-10-12-78-238 libra]# ls -l
total 48
...
lrwxrwxrwx. 1 root root   26 Feb 28 02:00 resource_limits.conf -> resource_limits.conf.large
...



Which is why ruby wouldn't work.