Bug 997786

Summary: oo-admin-upgrade failed to upgrade applications which have an version 'updated' cartridge
Product: OpenShift Online Reporter: Jianwei Hou <jhou>
Component: ContainersAssignee: Paul Morie <pmorie>
Status: CLOSED CURRENTRELEASE QA Contact: libra bugs <libra-bugs>
Severity: high Docs Contact:
Priority: high    
Version: 2.xCC: bmeng, dmace, xtian
Target Milestone: ---Keywords: TestBlocker
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-08-29 12:51:58 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Upgrade Error Log none

Description Jianwei Hou 2013-08-16 08:29:21 UTC
Created attachment 787199 [details]
Upgrade Error Log

Description of problem:
Some cartridges have their versions updated
phpmyadmin-3.4 -> phpmyadmin-3
jbosseap-6.0 -> jbosseap-6
jenkins-1.4 -> jenkins-1
jenkins-client-1.4 -> jenkins-1
switchyard-0.6 -> switchyard-0

When oo-admin-upgrade is processing the apps with above cartridges, it failed to upgrade them, according to the error log, the program is still referring to the obsolete version.

Here is one sample:
errors":["Unhandled exception performing step: key not found: (jenkins, 1.4, _)\n/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.13.5/lib/openshift-origin-node/model/cartridge_repository.rb:150:in `select'\n

Version-Release number of selected component (if applicable):
On devenv_3660

How reproducible:
Always

Steps to Reproduce:
1. Create all kinds of applications/cartridges on devenv-stage_442 ami
2. Upgrade it to latest(devenv_3660)
3. Execute necessary updates/configurations before the migration is performed:
oo-admin-clear-pending-ops --time 0
rhc-admin-migrate-datastore --non-compatible --version 2.0.32
4. [root@ip-10-151-21-209 oo-upgrade]# oo-admin-upgrade upgrade-node --upgrade-node ip-10-151-21-209 --version 2.0.32 --ignore-cartridge-version


Actual results:
Here is one sample, now we expect 'jbosseap-6.0' to be 'jbosseap-6', please see attachment for detail.

{"login":"jhou","app_name":"swithyardapp","gear_uuid":"520d91ab0012940b4d000007","version":"2.0.32","errors":["Gear upgrade result is marked incomplete","Failed upgrade after 2 tries"],"warnings":[],"times":{"time_upgrade_on_node_measured_from_broker":137,"time_total_upgrade_gear_measured_from_broker":1376635359438},"remote_upgrade_result":{"gear_uuid":"520d91ab0012940b4d000007","hostname":"ip-10-151-21-209","steps":{"inspect_gear_state":{"status":"complete","errors":[],"context":{}},"compute_itinerary":{"status":"incomplete","errors":["Unhandled exception performing step: key not found: (jbosseap, 6.0, _)\n/opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-node-1.13.5/lib/openshift-origin-node/model/cartridge_repository.rb:150:in `select'\n

Expected results:
Should refer to the new version of cartridges.

Additional info:

Comment 1 Paul Morie 2013-08-16 21:23:24 UTC
I believe the problem is that the gear upgrade extension needs to be configured in node.conf for this to work.  The following should be present in node.conf:

GEAR_UPGRADE_EXTENSION=/usr/libexec/openshift/lib/gear_upgrade_extension

Comment 3 Jianwei Hou 2013-08-19 10:05:17 UTC
Thanks, the migration is able to continue with comment 1 applied.