Red Hat Bugzilla – Bug 1007730
Ruby apps return 503 when accessing and cannot be started/restarted after migration
Last modified: 2015-05-14 19:28:32 EDT
Description of problem:
Create test data include ruby apps on devenv-stage_465, do mongo migration and node upgrade. All the started ruby apps cannot be accessed and cannot be started manually.
The app state shows the app is running.
# rhc app show ruby18 --state -pxx -k -email@example.com
Cartridge ruby-1.8, postgresql-9.2 is started
# curl -I http://ruby18-migration1.dev.rhcloud.com/
HTTP/1.1 503 Service Temporarily Unavailable
Date: Fri, 13 Sep 2013 07:39:21 GMT
Content-Type: text/html; charset=iso-8859-1
# rhc app restart ruby18 -p123123 -k -firstname.lastname@example.org
Unable to complete the requested operation due to: Failed to correctly execute all parallel operations.
Reference ID: bb8f0fe5a7945a4ba5d36adcf501f29e
Ruby apps on INT are also broken.
INT version: devenv_3779
Version-Release number of selected component (if applicable):
devenv-stage_465 to devenv-stage_471
Steps to Reproduce:
1. Launch devenv-stage_465 ami
2. Prepare test data
3. yum clean all && yum update -y
4. Clear broker cache
5. Restart service
6. Clear pending ops
7. Migrate datastore
# rhc admin-migrate-datastore --version 2.0.33 --non-compatible
8. Migrate gears
# oo-admin-upgrade upgrade-node --version 2.0.33 --ignore-cartridge-version
9. Clear broker cache
Ruby apps are broken as description.
Apps should work well after migration.
Found some maybe useful info in the development log:
2013-09-13 03:51:16.964 [DEBUG] DEBUG: server results: restarting Ruby cart (pid:30060)
2013-09-13 03:51:16.966 [DEBUG] DEBUG: server results: (pid:30060)
2013-09-13 03:51:16.971 [DEBUG] DEBUG: server results: /usr/lib/openshift/cartridge_sdk/bash/sdk: line 419: /var/lib/openshift/52327c15caf1de0aa3000004/ruby//etc/conf.d/passenv.conf: Permission denied (pid:30060)
2013-09-13 03:51:16.131 [ERROR] Failed to correctly execute all parallel operations (pid:30060)
Seems to me like the migration didn't run the 'setup' - I mean this line:
Meng, could you please confirm that this file exists on the filesystem?
If no, then it might be a migration problem (see Vojtech comment).
According to the http://openshift.github.io/documentation/oo_cartridge_developers_guide.html#bin-setup, the setup is run only on non-backward compatible upgrade.
I think we didn't mark the new cartridges incompatible in the manifest.yml, is that correct, Jhon?
The above statement would be related to the following cartridges:
Michal Fojtik, the file is not existing as Comment 2 mentioned.
Michal Fojtik, the file is existing on STG as Comment 2 mentioned, and in Comment 6 it means on INT.
Michal Fojtik, i think Comment 6&7 may make you confuse, sorry for that, please ignore them and refer to the following information:
After checking it on STG today, found that the ruby applications can be started/restarted and accessed successfully via browser. So, this issue just occurred on INT/devenv and the passenv.conf file is not on devenv/INT's filesystem.
According to above messages, lower the severity to medium. Thanks!
This should have been fixed in current STG(devenv-stage_472), please move it to ON_QA when you think it's ready.
Existing ruby apps work well on STG after migration.
Move bug to verified.