Bug 1018654

Summary: [deploy]Jenkins build will make app process restart when hot_deploy marker is enabled
Product: OpenShift Online Reporter: Lei Zhang <lzhang>
Component: ContainersAssignee: Andy Goldstein <agoldste>
Status: CLOSED CURRENTRELEASE QA Contact: libra bugs <libra-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 2.xCC: dmcphers, xtian
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-01-24 03:23:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Lei Zhang 2013-10-14 05:57:34 UTC
Description of problem:
When deploying a normal app with hot_deploy marker by jenkins, the app process will be restarted, but it will not restart when building without jenkins.


Before deploying:
[mypython26-chunchen.dev.rhcloud.com 525b66a0ef579cca530000ca]\> ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
1009     14550 14426  0 00:37 ?        00:00:00 sshd: 525b66a0ef579cca530000ca@pts/0
1009     14585 14550  0 00:37 pts/0    00:00:00 /bin/bash --init-file /usr/bin/rhcsh -i
1009     21955 14585  0 00:39 pts/0    00:00:00 ps -ef
1009     23673     1  0 Oct13 ?        00:00:00 /usr/sbin/httpd -C Include /var/lib/openshift/525b66a0ef579cca530000ca/
1009     23674 23673  0 Oct13 ?        00:00:00 /usr/sbin/rotatelogs /var/lib/openshift/525b66a0ef579cca530000ca/python
1009     23675 23673  0 Oct13 ?        00:00:00 /usr/sbin/rotatelogs /var/lib/openshift/525b66a0ef579cca530000ca/python
1009     23676 23673  0 Oct13 ?        00:00:00 /usr/sbin/httpd -C Include /var/lib/openshift/525b66a0ef579cca530000ca/
1009     23679 23673  0 Oct13 ?        00:00:00 /usr/sbin/httpd -C Include /var/lib/openshift/525b66a0ef579cca530000ca/
1009     23682 23673  0 Oct13 ?        00:00:00 /usr/sbin/httpd -C Include /var/lib/openshift/525b66a0ef579cca530000ca/
[mypython26-chunchen.dev.rhcloud.com 525b66a0ef579cca530000ca]\> find . -type f -name "*pid"|xargs cat
23673


After deploying:
[mypython26-chunchen.dev.rhcloud.com 525b66a0ef579cca530000ca]\> ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
1009      7007     1  0 00:44 ?        00:00:00 /usr/sbin/httpd -C Include /var/lib/openshift/525b66a0ef579cca530000ca/
1009      7011  7007  0 00:44 ?        00:00:00 /usr/sbin/rotatelogs /var/lib/openshift/525b66a0ef579cca530000ca/python
1009      7014  7007  0 00:44 ?        00:00:00 /usr/sbin/rotatelogs /var/lib/openshift/525b66a0ef579cca530000ca/python
1009      7020  7007  0 00:44 ?        00:00:00 /usr/sbin/httpd -C Include /var/lib/openshift/525b66a0ef579cca530000ca/
1009      7026  7007  0 00:44 ?        00:00:00 /usr/sbin/httpd -C Include /var/lib/openshift/525b66a0ef579cca530000ca/
1009      7030  7007  0 00:44 ?        00:00:00 /usr/sbin/httpd -C Include /var/lib/openshift/525b66a0ef579cca530000ca/
1009      7488 14585  0 00:44 pts/0    00:00:00 ps -ef
1009     14550 14426  0 00:37 ?        00:00:00 sshd: 525b66a0ef579cca530000ca@pts/0
1009     14585 14550  0 00:37 pts/0    00:00:00 /bin/bash --init-file /usr/bin/rhcsh -i
[mypython26-chunchen.dev.rhcloud.com 525b66a0ef579cca530000ca]\> find . -type f -name "*pid"|xargs cat
7007


Version-Release number of selected component (if applicable):
devenv_3893

How reproducible:
always

Steps to Reproduce:
1.create one app with jenkin
#rhc app create mypython26 python-2.6 --enable-jenkins
2.change something in local repo and git push, check the pid of app process in rhcsh
3.remove jenkins-client cartridge 
#rhc cartridge remove -a mypython26 -c jenkins-client-1
4.change something in local repo and git push, check the pid of app process in rhcsh

Actual results:
Found the app process will restart when deploying with hot_deploy marker by jenkins.

Expected results:
The app process should not restart when deploying with hot_deploy marker by jenkins.

Additional info:

Comment 1 Lei Zhang 2013-10-14 06:00:02 UTC
Started by user Jenkins System Builder
Building remotely on mypython26bldr in workspace /var/lib/openshift/525b86866474637f730000cc/app-root/runtime/repo
Checkout:repo / /var/lib/openshift/525b86866474637f730000cc/app-root/runtime/repo - hudson.remoting.Channel@138202d:mypython26bldr
Using strategy: Default
Checkout:repo / /var/lib/openshift/525b86866474637f730000cc/app-root/runtime/repo - hudson.remoting.LocalChannel@10e215c
Cloning the remote Git repository
Cloning repository origin
Fetching upstream changes from ssh://525b85a664746342000001d4.rhcloud.com/~/git/mypython26.git
Seen branch in repository origin/HEAD
Seen branch in repository origin/master
Commencing build of Revision 8d2e995816ed1abbef2ef888f6a096ae2d61463c (origin/HEAD, origin/master)
Checking out Revision 8d2e995816ed1abbef2ef888f6a096ae2d61463c (origin/HEAD, origin/master)
No change to record in branch origin/HEAD
No change to record in branch origin/master
[repo] $ /bin/sh -xe /tmp/hudson6714936184699176765.sh
+ alias 'rsync=rsync --delete-after -azO -e '\''/usr/libexec/openshift/cartridges/jenkins/bin/git_ssh_wrapper.sh'\'''
+ upstream_ssh=525b85a664746342000001d4.rhcloud.com
+ set +e
+ rsync --delete-after -azO -e /usr/libexec/openshift/cartridges/jenkins/bin/git_ssh_wrapper.sh '525b85a664746342000001d4.rhcloud.com:$OPENSHIFT_BUILD_DEPENDENCIES_DIR' /var/lib/openshift/525b86866474637f730000cc/app-root/runtime/build-dependencies/
+ rsync --delete-after -azO -e /usr/libexec/openshift/cartridges/jenkins/bin/git_ssh_wrapper.sh '525b85a664746342000001d4.rhcloud.com:${OPENSHIFT_DEPENDENCIES_DIR}' /var/lib/openshift/525b86866474637f730000cc/app-root/runtime/dependencies/
+ set -e
+ gear build
running develop
running egg_info
creating YourAppName.egg-info
writing YourAppName.egg-info/PKG-INFO
writing top-level names to YourAppName.egg-info/top_level.txt
writing dependency_links to YourAppName.egg-info/dependency_links.txt
writing YourAppName.egg-info/PKG-INFO
writing top-level names to YourAppName.egg-info/top_level.txt
writing dependency_links to YourAppName.egg-info/dependency_links.txt
writing manifest file 'YourAppName.egg-info/SOURCES.txt'
reading manifest file 'YourAppName.egg-info/SOURCES.txt'
writing manifest file 'YourAppName.egg-info/SOURCES.txt'
running build_ext
Creating /var/lib/openshift/525b86866474637f730000cc/app-deployments/2013-10-14_01-52-10.286/dependencies/virtenv/lib/python2.6/site-packages/YourAppName.egg-link (link to .)
Adding YourAppName 1.0 to easy-install.pth file

Installed /var/lib/openshift/525b86866474637f730000cc/app-root/runtime/repo
Processing dependencies for YourAppName==1.0
Finished processing dependencies for YourAppName==1.0
Script /var/lib/openshift/525b86866474637f730000cc/python//virtenv/bin/activate.csh cannot be made relative (it's not a normal script that starts with #!/var/lib/openshift/525b86866474637f730000cc/python/virtenv/bin/python)
Script /var/lib/openshift/525b86866474637f730000cc/python//virtenv/bin/activate.fish cannot be made relative (it's not a normal script that starts with #!/var/lib/openshift/525b86866474637f730000cc/python/virtenv/bin/python)
+ /usr/libexec/openshift/cartridges/jenkins/bin/git_ssh_wrapper.sh 525b85a664746342000001d4.rhcloud.com 'gear stop --conditional --exclude-web-proxy --git-ref 8d2e995816ed1abbef2ef888f6a096ae2d61463c'
Skipping gear stop due to presence of hot deploy marker
++ /usr/libexec/openshift/cartridges/jenkins/bin/git_ssh_wrapper.sh 525b85a664746342000001d4.rhcloud.com 'gear create-deployment-dir'
+ deployment_dir=2013-10-14_01-53-52.788
+ rsync --delete-after -azO -e /usr/libexec/openshift/cartridges/jenkins/bin/git_ssh_wrapper.sh /var/lib/openshift/525b86866474637f730000cc/app-root/runtime/repo/ 525b85a664746342000001d4.rhcloud.com:app-deployments/2013-10-14_01-53-52.788/repo/
+ rsync --delete-after -azO -e /usr/libexec/openshift/cartridges/jenkins/bin/git_ssh_wrapper.sh /var/lib/openshift/525b86866474637f730000cc/app-root/runtime/build-dependencies/ 525b85a664746342000001d4.rhcloud.com:app-deployments/2013-10-14_01-53-52.788/build-dependencies/
+ rsync --delete-after -azO -e /usr/libexec/openshift/cartridges/jenkins/bin/git_ssh_wrapper.sh /var/lib/openshift/525b86866474637f730000cc/app-root/runtime/dependencies/ 525b85a664746342000001d4.rhcloud.com:app-deployments/2013-10-14_01-53-52.788/dependencies/
+ /usr/libexec/openshift/cartridges/jenkins/bin/git_ssh_wrapper.sh 525b85a664746342000001d4.rhcloud.com 'gear remotedeploy --deployment-datetime 2013-10-14_01-53-52.788'
Preparing build for deployment
Prepared deployment artifacts in /var/lib/openshift/525b85a664746342000001d4/app-deployments/2013-10-14_01-53-52.788
Deployment id is 9a05a366
Activating deployment
Stopping PYTHON cart
Syntax error on line 4 of /var/lib/openshift/525b85a664746342000001d4/python/etc/conf.d/openshift.conf:
DocumentRoot must be a directory
Script /var/lib/openshift/525b85a664746342000001d4/python//virtenv/bin/activate.csh cannot be made relative (it's not a normal script that starts with #!/var/lib/openshift/525b85a664746342000001d4/python/virtenv/bin/python)
Script /var/lib/openshift/525b85a664746342000001d4/python//virtenv/bin/activate.fish cannot be made relative (it's not a normal script that starts with #!/var/lib/openshift/525b85a664746342000001d4/python/virtenv/bin/python)
Starting PYTHON cart
Archiving artifacts
Finished: SUCCESS

Comment 2 Andy Goldstein 2013-10-23 12:21:49 UTC
https://github.com/openshift/origin-server/pull/3955

Comment 3 Lei Zhang 2013-10-24 05:24:02 UTC
Verified on devenv_3937, process restart when jenkin build with hot_deploy marker enable. 

Before build check PID of python httpd:
[mypython26-chunchen.dev.rhcloud.com 5268a922fa7baba62f00001e]\> find . -type f -name "*pid"|xargs cat 
4390

After build check PID of python httpd:
[mypython26-chunchen.dev.rhcloud.com 5268a922fa7baba62f00001e]\> find . -type f -name "*pid"|xargs cat 
4390