Bug 785550

Summary: Jenkins job was seen taking ~5 mins
Product: OKD Reporter: Linqing Lu <lilu>
Component: ContainersAssignee: Dan McPherson <dmcphers>
Status: CLOSED CURRENTRELEASE QA Contact: libra bugs <libra-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 2.xCC: bmeng, gshipley
Target Milestone: ---Keywords: Regression, TestBlocker
Target Release: ---   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: devenv_1567 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-02-02 23:55:37 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Linqing Lu 2012-01-29 14:08:00 UTC
Description of problem:
jenkins job always hang there (confirmed >30min) when git push

Version-Release number of selected component (if applicable):
devenv_1564/1565

How reproducible:
always

Steps to Reproduce:
1. create an app
2. create an jenkins server
3. embed jenkins-client into the app
4. do some minor modification and git push
  
Actual results:
job hang there forever at "Waiting for build to schedule........."

Expected results:
jenkins job finishes as expected

Additional info:

Comment 1 Meng Bo 2012-01-30 09:45:46 UTC
check on devenv_1567,
the jenkins jobs will hang there about 5 mins, then finish building.
and we think it should not be so long.
Assign it back and please help on this.

[bmeng@localhost ruby1]$ git push
Counting objects: 3, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (2/2), 242 bytes, done.
Total 2 (delta 1), reused 0 (delta 0)
remote: Executing Jenkins build.
remote: 
remote: You can track your build at http://jenkins-bmengdev1.dev.rhcloud.com/job/ruby1-build
remote: 
remote: Waiting for build to schedule.......................................................................................................................................................................................................................................................................................................................................................................................................................................................Done
remote: Waiting for job to complete..........................Done
remote: SUCCESS
remote: New build has been deployed.
To ssh://8d300be27de74648816f56f98575c336.rhcloud.com/~/git/ruby1.git/
   6676589..472b2c9  master -> master

Comment 2 Meng Bo 2012-01-30 09:57:15 UTC
attach the log of the job.

Started by user system_builder
Started by user admin
Building remotely on ruby1bldr
Checkout:ruby1-build / ruby1bldr/ci/jenkins/workspace/ruby1-build - hudson.remoting.Channel@4690555f:ruby1bldr
Using strategy: Default
Checkout:ruby1-build / ruby1bldr/ci/jenkins/workspace/ruby1-build - hudson.remoting.LocalChannel@5088f1c5
Cloning the remote Git repository
Cloning repository origin
Fetching upstream changes from ssh://8d300be27de74648816f56f98575c336.rhcloud.com/~/git/ruby1.git/
Seen branch in repository origin/HEAD
Seen branch in repository origin/master
Commencing build of Revision 472b2c901482aae8bf0a3ca4810c46e3665de5ce (origin/HEAD, origin/master)
Checking out Revision 472b2c901482aae8bf0a3ca4810c46e3665de5ce (origin/HEAD, origin/master)
No change to record in branch origin/HEAD
No change to record in branch origin/master
[ruby1-build] $ /bin/sh -xe /tmp/hudson5893484858083995668.sh
+ alias 'rsync=rsync --delete-after -az -e '\''/usr/libexec/li/cartridges/jenkins-1.4/info/bin/git_ssh_wrapper.sh'\'''
+ . ci_build.sh
++ set +x
Running .openshift/action_hooks/pre_build
Running .openshift/action_hooks/build
+ '[' -f Gemfile ']'
+ /usr/libexec/li/cartridges/jenkins-1.4/info/bin/git_ssh_wrapper.sh 8d300be27de74648816f56f98575c336.rhcloud.com 'ctl_all stop'
Could not create directory '/var/lib/libra/208fef91d2a24ea5b55625e96c6e2c21/.ssh'.
WARNING: This ssh terminal was started without a tty.
          It is highly recommended to login with: ssh -t
+ /var/lib/libra/8d300be27de74648816f56f98575c336/ruby1/ruby1_ctl.sh stop
Waiting for stop to finish
Waiting for stop to finish
Waiting for stop to finish
+ rsync --delete-after -az -e /usr/libexec/li/cartridges/jenkins-1.4/info/bin/git_ssh_wrapper.sh /var/lib/libra/208fef91d2a24ea5b55625e96c6e2c21/ruby1bldr/ci/jenkins/workspace/ruby1-build/ '8d300be27de74648816f56f98575c336.rhcloud.com:~/ruby1/repo/'
Could not create directory '/var/lib/libra/208fef91d2a24ea5b55625e96c6e2c21/.ssh'.
+ /usr/libexec/li/cartridges/jenkins-1.4/info/bin/git_ssh_wrapper.sh 8d300be27de74648816f56f98575c336.rhcloud.com deploy.sh
Could not create directory '/var/lib/libra/208fef91d2a24ea5b55625e96c6e2c21/.ssh'.
Running .openshift/action_hooks/deploy
+ /usr/libexec/li/cartridges/jenkins-1.4/info/bin/git_ssh_wrapper.sh 8d300be27de74648816f56f98575c336.rhcloud.com 'ctl_all start'
Could not create directory '/var/lib/libra/208fef91d2a24ea5b55625e96c6e2c21/.ssh'.
WARNING: This ssh terminal was started without a tty.
          It is highly recommended to login with: ssh -t
+ /var/lib/libra/8d300be27de74648816f56f98575c336/ruby1/ruby1_ctl.sh start
+ /usr/libexec/li/cartridges/jenkins-1.4/info/bin/git_ssh_wrapper.sh 8d300be27de74648816f56f98575c336.rhcloud.com post_deploy.sh
Could not create directory '/var/lib/libra/208fef91d2a24ea5b55625e96c6e2c21/.ssh'.
Running .openshift/action_hooks/post_deploy
Archiving artifacts
Finished: SUCCESS

Comment 3 Dan McPherson 2012-01-30 15:05:41 UTC
Lowering the priority and severity based on a different problem being reported.

I couldn't recreate the delay.  To have an idea what's going on I would need to see the log under:

http://appname-namespace.dev.rhcloud.com/log/all for the jenkins app.

My guess would be DNS was slow to resolve.

Comment 4 Meng Bo 2012-01-31 03:14:53 UTC
try it again on devenv_1567,
and cannot recreate the issue too...

Mark this bug as VERIFIED.

Comment 5 Mrunal Patel 2012-02-02 23:55:37 UTC
Closing verified bugs.