Bug 966952

Summary:	Can not add/remove cartridge or scale up/down for migrated apps on INT
Product:	OpenShift Online	Reporter:	Jianwei Hou <jhou>
Component:	Containers	Assignee:	Dan McPherson <dmcphers>
Status:	CLOSED CURRENTRELEASE	QA Contact:	libra bugs <libra-bugs>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	2.x	CC:	qgong, xtian
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2013-06-11 04:10:55 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Jianwei Hou 2013-05-24 11:29:39 UTC

Description of problem:
Try to add/remove cartridge from an app or try to scale up/down an app on INT, the broker returns "Your application is being upgraded and configuration changes can not be made at this time.  Please try again later."
Now that the migration is run on INT, this problem is not expected, maybe the following apps failed to migrate?

----list of failed apps----
| app name  | UUID                     |
| r18       | 51958d736cec0e68f600006b |
| diy1      | 51958ae803ef64652b00014f |
| wordpress | 51958a5e03ef64652b00010c |
| rails     | 51958d9a6cec0e68f6000081 |
| jbossas1  | 51958c5a03ef64867f0000bd |
| jbossas1s | 5195907b6cec0e7ba80003ab |
| python33  | 51958e5403ef649090000017 |
| jbossews2 | 51958f3303ef649090000075 |
| jbossews1s| 5195958703ef64413000007a |
| python26s | 519592956cec0e68f6000166 |
| php1s     | 519591376cec0e7ba80003eb |
| python27s | 5195921d03ef64867f00018a |
| ruby18    | 519ae0076cec0e1c800000ba |
| perl1s    | 519591a76cec0e68f600012a |
| ews2s     | 519b256403ef64a198000001 |
| jbosseap1s| 5195950303ef64413000003b |


Version-Release number of selected component (if applicable):
On INT(devenv_3266)

How reproducible:
Always

Steps to Reproduce:
1. Prepare test data in v1 format before INT update
2. After INT is upgraded and migrated, add/remove cartridge to an app
3. Scale up/down a scalable app

Actual results:
During step 2 and step 3, most applications will get "Your application is being upgraded and configuration changes can not be made at this time.  Please try again later."
This message is not expected to see if migration was successful

Expected results:
Should not see this error

Additional info:

Comment 1 Dan McPherson 2013-05-24 18:45:21 UTC

We should have asked ops to not run migrate-mongo-2.0.28 more than once.  Will split out the script get resolved with ops.

Comment 2 Dan McPherson 2013-05-24 19:53:37 UTC

This should be fixed.  Please give it a try.

Comment 3 Jianwei Hou 2013-05-27 02:12:04 UTC

This is verified  and fixed on INT, the above apps are allowed to add/remove cartridges and can be scaled up and down.

Comment 4 Rony Gong 🔥 2013-05-29 06:28:31 UTC

Retest it on INT(devenv_3281) server  --Failed
Can't add db cartridge to some migration application eg: jbosseap
[qgong@dhcp-10-94 webapp]$ addmys q2jea
Adding mysql-5.1 to application 'q2jea' ... Your application is being upgraded and configuration changes can not be made at this time.  Please try again later.

Comment 5 Rony Gong 🔥 2013-05-29 07:09:37 UTC

And I collect below cartridges also can't embed db cartridge well eg:   ruby-1.8/1.9, python-2.7/3.3, zend-5.6, jbosseap-6.0, jbossas-7

one of the jbosseap application's detail info:

DNS: http://qjea-qgong1.int.rhcloud.com/
UUID: 519595746cec0efab4000072
SSH: 519595746cec0efab4000072.rhcloud.com

Comment 6 Dan McPherson 2013-05-29 14:06:30 UTC

I am tracking this down but I think ops reran the quarantine script.  They have now unquarantined everything again.  Let me know if you see the issue again.

Comment 7 Jianwei Hou 2013-05-30 03:02:49 UTC

Verified this on INT, now the apps are back to normal again.