Bug 1025267

Summary:	Met error "jbossews//bin/build: line 33: force_clean_build_enabled_for_latest_deployment command not found" when push changes to existing apps after migration.
Product:	OpenShift Online	Reporter:	Liang Xia <lxia>
Component:	Containers	Assignee:	Dan Mace <dmace>
Status:	CLOSED CURRENTRELEASE	QA Contact:	libra bugs <libra-bugs>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	2.x	CC:	dmace, jhou
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2014-01-24 03:28:17 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Liang Xia 2013-10-31 10:54:04 UTC

Description of problem:
When push changes to existing apps after upgrade and migration, met error:
remote: /var/lib/openshift/5271c0a25c116dab12000084/jbossews//bin/build: line 33: force_clean_build_enabled_for_latest_deployment: command not found
Also met this error when snapshot restore the app.

Version-Release number of selected component (if applicable):
migrate from devenv-stage_528 to devenv_3973

How reproducible:
always

Steps to Reproduce:
1. Create jbossews-2.0 apps with cron-1.4 mysql-5.1 postgresql-9.2 mongodb-2.2 phpmyadmin-4 rockmongo-1.1 metrics-0.1 embedded on devenv-stage_528.
2. Upgrade the devent-stage instance to latest as devenv_3973.
3. After upgrade, clear broker cache, restart broker and mcollective services
4. Do datastore migration
# rhc-admin-migrate-datastore --compatible --version 2.0.35
5. Do gear migration
# oo-admin-upgrade upgrade-node --version 2.0.35 --ignore-cartridge-version
6.change something in the app's local repo and git push.

Actual results:
remote: Stopping jbossews cartridge
remote: Sending SIGTERM to jboss:1593 ...
remote: kill -9 1982
remote: kill -9 1593
remote: Stopping PHPMyAdmin cartridge
remote: Waiting for stop to finish
remote: Stopping Cron cartridge
remote: Stopping metrics cartridge
remote: [Thu Oct 31 06:26:56 2013] [warn] module php5_module is already loaded, skipping
remote: Waiting for stop to finish
remote: Stopping MongoDB cartridge
remote: waiting for server to shut down.... done
remote: server stopped
remote: Stopping Postgres cartridge
remote: Stopping RockMongo cartridge
remote: [Thu Oct 31 06:26:59 2013] [warn] module php5_module is already loaded, skipping
remote: Waiting for stop to finish
remote: Stopping MySQL cartridge
remote: /var/lib/openshift/5271c0a25c116dab12000084/jbossews//bin/build: line 33: force_clean_build_enabled_for_latest_deployment: command not found
remote: Using Maven mirror /var/lib/openshift/5271c0a25c116dab12000084/app-root/runtime/repo//.openshift/config/settings.rhcloud.xml
remote: Apache Maven 3.0.3 (r1075437; 2011-06-20 13:22:37-0400)
...

Expected results:
Git push successfully without errors

Additional info:

Comment 1 Dan Mace 2013-10-31 15:46:38 UTC

Couldn't reproduce with a fresh upgrade from devenv_3974 to devenv-stage_532. Examining the yum log of ec2-54-205-26-133.compute-1.amazonaws.com leads me to believe it wasn't upgraded to the new RPMs properly.

If this issue can be reproduced, please:

1. Tag the test VM(s) with -preserve
2. Specify the VM(s) in the bz

Thanks!

Comment 2 Liang Xia 2013-11-01 05:43:28 UTC

Tried again with following scenarios:

non-scalable jbossews-2.0 without force_clean_build marker.
non-scalable jbossews-2.0 with force_clean_build marker.
scalable jbossews-2.0 without force_clean_build marker.
scalable jbossews-2.0 with force_clean_build marker.

Steps:
1. Create apps on devenv-stage_528.
2. Upgrade the devent-stage instance to latest as devenv_3978.
3. After upgrade, clear broker cache, restart broker and mcollective services
4. Do datastore migration
# rhc-admin-migrate-datastore --compatible --version 2.0.35
5. Do gear migration
# oo-admin-upgrade upgrade-node --version 2.0.35 --ignore-cartridge-version
6.change something in the app's local repo and git push.

None of the scenarios met the error.

Move to verified.

Comment 3 Jianwei Hou 2013-11-01 12:07:15 UTC

Also tested after migration on devenv-stage_546, the issue is not reproduced.