With the new deployment code, we need to make sure that existing Jenkins jobs continue to function after the upgrade happens. For example: - create app, enable jenkins - make sure the build works - upgrade system to new deploy code - push a change to the app - make sure the build continues to work
*** Bug 1020838 has been marked as a duplicate of this bug. ***
This can be reproduced on INT(devenv_3942). Description of problem: When pushing changes to existing app after server migration, met following errors: remote: An error occurred executing 'gear postreceive' (exit code: 1) remote: Error message: Failed to execute: 'control post-receive' for /var/lib/openshift/5264c51403ef642922000319/jenkins-client Version-Release number of selected component (if applicable): INT(devenv_3942) How reproducible: always Steps to Reproduce: 1.create an jbossews-2.0 app with jenkins-client. 2.migrate the server. 3.make some change in the local repo and git push. Actual results: remote: Executing Jenkins build. remote: remote: You can track your build at https://jenkins-lxiamigrate.int.rhcloud.com/job/jbossews2-build remote: remote: Waiting for build to schedule.....................................................Done remote: Waiting for job to complete....................................................Done remote: FAILED remote: !!!!!!!! remote: Deployment Halted! remote: If the build failed before the deploy step, your previous remote: build is still running. Otherwise, your application may be remote: partially deployed or inaccessible. remote: Fix the build and try again. remote: !!!!!!!! remote: An error occurred executing 'gear postreceive' (exit code: 1) remote: Error message: Failed to execute: 'control post-receive' for /var/lib/openshift/5264c51403ef642922000319/jenkins-client remote: remote: For more details about the problem, try running the command again with the '--trace' option. To ssh://5264c51403ef642922000319.rhcloud.com/~/git/jbossews2.git/ c5bba71..5454799 master -> master Expected results: Changes deployed successfully without errors.
https://github.com/openshift/origin-server/pull/4003
Current INT is devenv_3948. We will check it when next INT ready.
Jenkins build after migration with current devenv_3953 still fails. [openshift@dhcp-10-143 jbossews1]$ vi src/main/webapp/index.html [openshift@dhcp-10-143 jbossews1]$ gitpush [master 45b178e] this is a comment Committer: openshift <openshift.redhat.com> Your name and email address were configured automatically based on your username and hostname. Please check that they are accurate. You can suppress this message by setting them explicitly: git config --global user.name "Your Name" git config --global user.email you After doing this, you may fix the identity used for this commit with: git commit --amend --reset-author 1 file changed, 1 insertion(+), 1 deletion(-) warning: push.default is unset; its implicit value is changing in Git 2.0 from 'matching' to 'simple'. To squelch this message and maintain the current behavior after the default changes, use: git config --global push.default matching To squelch this message and adopt the new behavior now, use: git config --global push.default simple See 'git help config' and search for 'push.default' for further information. (the 'simple' mode was introduced in Git 1.7.11. Use the similar mode 'current' instead of 'simple' if you sometimes use older versions of Git) @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: POSSIBLE DNS SPOOFING DETECTED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ The RSA host key for jbossews1-migration1.dev.rhcloud.com has changed, and the key for the corresponding IP address 174.129.73.153 is unchanged. This could either mean that DNS SPOOFING is happening or the IP address for the host and its host key have changed at the same time. Offending key for IP in /home/openshift/.ssh/known_hosts:199 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has just been changed. The fingerprint for the RSA key sent by the remote host is 0b:9d:8d:1e:a5:30:cc:6c:9c:39:80:0f:51:34:cf:a5. Please contact your system administrator. Add correct host key in /home/openshift/.ssh/known_hosts to get rid of this message. Offending RSA key in /home/openshift/.ssh/known_hosts:47 Password authentication is disabled to avoid man-in-the-middle attacks. Keyboard-interactive authentication is disabled to avoid man-in-the-middle attacks. Counting objects: 11, done. Compressing objects: 100% (5/5), done. Writing objects: 100% (6/6), 470 bytes | 0 bytes/s, done. Total 6 (delta 4), reused 0 (delta 0) remote: Executing Jenkins build. remote: remote: You can track your build at https://jenkins1-migration1.dev.rhcloud.com/job/jbossews1-build remote: remote: Waiting for build to schedule......................................Done remote: Waiting for job to complete....................................................................................Done remote: FAILED remote: !!!!!!!! remote: Deployment Halted! remote: If the build failed before the deploy step, your previous remote: build is still running. Otherwise, your application may be remote: partially deployed or inaccessible. remote: Fix the build and try again. remote: !!!!!!!! remote: An error occurred executing 'gear postreceive' (exit code: 1) remote: Error message: Failed to execute: 'control post-receive' for /var/lib/openshift/526dc7e5cab0818f3500017f/jenkins-client remote: remote: For more details about the problem, try running the command again with the '--trace' option. To ssh://526dc7e5cab0818f3500017f.rhcloud.com/~/git/jbossews1.git/ 9bb1dfa..45b178e master -> master Mark the bug as assigned.
Checked on latest INT (devenv_3957), found the following error in log for scaling app. Stopping gear... HAProxy instance is stopped Stopping jbosseap cartridge Sending SIGTERM to jboss:32113 ... + rsync --delete-after -azO -e /usr/libexec/openshift/cartridges/jenkins/bin/git_ssh_wrapper.sh /var/lib/openshift/526f4f0303ef646f000002a2//.m2/ '5264d33003ef64b3a8000174.rhcloud.com:~/.m2/' + rsync --delete-after -azO -e /usr/libexec/openshift/cartridges/jenkins/bin/git_ssh_wrapper.sh /var/lib/openshift/526f4f0303ef646f000002a2/app-root/runtime/repo/deployments/ '5264d33003ef64b3a8000174.rhcloud.com:${OPENSHIFT_REPO_DIR}deployments/' + rsync --delete-after -azO -e /usr/libexec/openshift/cartridges/jenkins/bin/git_ssh_wrapper.sh /var/lib/openshift/526f4f0303ef646f000002a2/app-root/runtime/repo/.openshift/ '5264d33003ef64b3a8000174.rhcloud.com:${OPENSHIFT_REPO_DIR}.openshift/' + /usr/libexec/openshift/cartridges/jenkins/bin/git_ssh_wrapper.sh 5264d33003ef64b3a8000174.rhcloud.com 'gear remotedeploy' Preparing build for deployment deployment_datetime is required Build step 'Execute shell' marked build as failure Archiving artifacts Finished: FAILURE
I have tested PHP and JBoss EWS applications and the Jenkins builds work fine using the previous jenkins_shell_command. What I do is this: 1) create a new application in the latest devenv 2) add the jenkins-client cartridge 3) log in to Jenkins 4) edit the Jenkins job and replace its contents with those from e.g. https://github.com/openshift/origin-server/blob/af3621a263675b16eb4781caa0f44c28054992df/cartridges/openshift-origin-cartridge-jbossews/metadata/jenkins_shell_command.erb (but make sure you keep the upstream_ssh line from the job so it has the right URL) 5) perform a build At this point, it should work. If you see this: Preparing build for deployment deployment_datetime is required <------ it means that the upgraded gear is either missing the app-deployments/current symlink or it is a broken symlink. I looked at 5264d33003ef64b3a8000174.rhcloud.com and found that the "current" symlink is in fact broken. I also noticed that the app is a few days old (from around 10/24) and would recommend retesting with a new app that gets upgraded because we've fixed several upgrade bugs since then. I believe that the Jenkins builds will work fine after an upgrade once all upgrade issues have been fixed.
Checked on devenv_3966 which upgraded from devenv-stage_528, Jenkins build for both scale and non-scale apps are working fine now. Move bug to verified.