Bug 970840
| Summary: | Gear fails to migrate if low on disk space | ||
|---|---|---|---|
| Product: | OpenShift Online | Reporter: | Wesley Hearn <whearn> |
| Component: | Containers | Assignee: | Dan McPherson <dmcphers> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | libra bugs <libra-bugs> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 2.x | CC: | bmeng, dmcphers, jhonce |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2013-06-11 04:17:04 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Checked on devenv-stage_360, issue fixed.
1. Prepare app with quota full filled on devenv-stage_353.
2. Do upgrade and migration.
3. App migration successfully.
Migrating app 'quota' gear 'quota' with uuid '689acd84ce6b11e28c1222000a8d945a' on node 'ip-10-141-148-90' for user: bmeng
Migrating gear on node with: /usr/bin/rhc-admin-migrate --login 'bmeng' --migrate-gear '689acd84ce6b11e28c1222000a8d945a' --app-name 'quota' --version '2.0.28'
Migrating on node...
***time_migrate_on_node_measured_from_broker=29946***
Migrate on node output:
Beginning V1 -> V2 migration
Inspecting gear at /var/lib/openshift/689acd84ce6b11e28c1222000a8d945a
Pre-migration state: started
Marking step inspect_gear_state complete
Marking step detect_switchyard complete
Beginning quota blocks: 1048576 inodes: 40000
Checking for V1 stop lock in ["php-5.3"]
V1 stop lock not detected
Marking step migrate_stop_lock complete
Stopping gear with uuid '689acd84ce6b11e28c1222000a8d945a' on node 'ip-10-141-148-90'
Marking step stop_gear complete
Marking step pam_nproc_soft complete
Marking step gear_env_cleanup complete
Migrated TMP v1 value [export TMP='/tmp/'] to raw value [/tmp/]
Migrated HOME v1 value [export HOME='/var/lib/openshift/689acd84ce6b11e28c1222000a8d945a/'] to raw value [/var/lib/openshift/689acd84ce6b11e28c1222000a8d945a/]
Migrated OPENSHIFT_REPO_DIR v1 value [export OPENSHIFT_REPO_DIR='/var/lib/openshift/689acd84ce6b11e28c1222000a8d945a/app-root/runtime/repo/'] to raw value [/var/lib/openshift/689acd84ce6b11e28c1222000a8d945a/app-root/runtime/repo/]
Migrated OPENSHIFT_INTERNAL_PORT v1 value [export OPENSHIFT_INTERNAL_PORT='8080'] to raw value [8080]
Migrated OPENSHIFT_APP_DNS v1 value [export OPENSHIFT_APP_DNS='quota-bmeng.dev.rhcloud.com'] to raw value [quota-bmeng.dev.rhcloud.com]
Migrated OPENSHIFT_GEAR_DNS v1 value [export OPENSHIFT_GEAR_DNS='quota-bmeng.dev.rhcloud.com'] to raw value [quota-bmeng.dev.rhcloud.com]
Migrated OPENSHIFT_HOMEDIR v1 value [export OPENSHIFT_HOMEDIR='/var/lib/openshift/689acd84ce6b11e28c1222000a8d945a/'] to raw value [/var/lib/openshift/689acd84ce6b11e28c1222000a8d945a/]
Migrated TMPDIR v1 value [export TMPDIR='/tmp/'] to raw value [/tmp/]
Migrated OPENSHIFT_APP_UUID v1 value [export OPENSHIFT_APP_UUID='689acd84ce6b11e28c1222000a8d945a'] to raw value [689acd84ce6b11e28c1222000a8d945a]
Migrated OPENSHIFT_PHP_IP v1 value [export OPENSHIFT_PHP_IP='127.0.253.129'] to raw value [127.0.253.129]
Migrated HISTFILE v1 value [export HISTFILE='/var/lib/openshift/689acd84ce6b11e28c1222000a8d945a/app-root/data/.bash_history'] to raw value [/var/lib/openshift/689acd84ce6b11e28c1222000a8d945a/app-root/data/.bash_history]
Migrated TMP_DIR v1 value [export TMP_DIR='/tmp/'] to raw value [/tmp/]
Migrated OPENSHIFT_TMP_DIR v1 value [export OPENSHIFT_TMP_DIR='/tmp/'] to raw value [/tmp/]
Migrated OPENSHIFT_APP_NAME v1 value [export OPENSHIFT_APP_NAME='quota'] to raw value [quota]
Migrated OPENSHIFT_PHP_PORT v1 value [export OPENSHIFT_PHP_PORT='8080'] to raw value [8080]
Migrated OPENSHIFT_INTERNAL_IP v1 value [export OPENSHIFT_INTERNAL_IP='127.0.253.129'] to raw value [127.0.253.129]
Migrated OPENSHIFT_PHP_LOG_DIR v1 value [export OPENSHIFT_PHP_LOG_DIR='/var/lib/openshift/689acd84ce6b11e28c1222000a8d945a/php-5.3/logs/'] to raw value [/var/lib/openshift/689acd84ce6b11e28c1222000a8d945a/php-5.3/logs/]
Migrated OPENSHIFT_GEAR_UUID v1 value [export OPENSHIFT_GEAR_UUID='689acd84ce6b11e28c1222000a8d945a'] to raw value [689acd84ce6b11e28c1222000a8d945a]
Migrated OPENSHIFT_DATA_DIR v1 value [export OPENSHIFT_DATA_DIR='/var/lib/openshift/689acd84ce6b11e28c1222000a8d945a/app-root/data/'] to raw value [/var/lib/openshift/689acd84ce6b11e28c1222000a8d945a/app-root/data/]
Migrated OPENSHIFT_GEAR_NAME v1 value [export OPENSHIFT_GEAR_NAME='quota'] to raw value [quota]
Migrated PATH v1 value [export PATH=/usr/libexec/openshift/cartridges/php-5.3/info/bin/:/usr/libexec/openshift/cartridges/abstract-httpd/info/bin/:/usr/libexec/openshift/cartridges/abstract/info/bin/:/bin:/usr/bin:/sbin:/usr/sbin] to raw value [/usr/libexec/openshift/cartridges/php-5.3/info/bin/:/usr/libexec/openshift/cartridges/abstract-httpd/info/bin/:/usr/libexec/openshift/cartridges/abstract/info/bin/:/bin:/usr/bin:/sbin:/usr/sbin]
Marking step env_vars_to_raw complete
Migrating TYPELESS_TRANSLATED_VARS to discrete variables
Marking step typeless_translated_vars complete
Marking step relocate_uservars complete
Marking step reconfigure_git_repo complete
Carts to migrate: {:framework_carts=>["php-5.3"]}
Migrating cartridge php
Marking step php_create_directory complete
Marking step php_setup complete
Marking step php_erb complete
applying php-5.3 migration post-process
Moving contents of php-5.3/logs to php/logs
Moving /var/lib/openshift/689acd84ce6b11e28c1222000a8d945a/php-5.3/logs/error_log-20130606-000000-EST
Moving contents of php-5.3/sessions to php/sessions
Moving contents of php-5.3/phplib to php/phplib
Moving /var/lib/openshift/689acd84ce6b11e28c1222000a8d945a/php-5.3/phplib/pear
Marking step php_hook complete
Marking step php_ownership complete
Marking step php_connect_frontend complete
0
Starting gear with uuid '689acd84ce6b11e28c1222000a8d945a' on node 'ip-10-141-148-90'
Start gear output: CLIENT_MESSAGE: Starting Apache+mod_php HTTPD server
Marking step start_gear complete
Validating gear 689acd84ce6b11e28c1222000a8d945a post-migration
Pre-migration state: started
Post-migration response code: 200
Marking step validate_gear complete
Cleaning up after migration
***time_migrate_on_node_measured_from_node=29694***
Resetting quota blocks: 1048576 inodes: 40000
After migration, check disk quota, still full
[quota-bmeng.dev.rhcloud.com 689acd84ce6b11e28c1222000a8d945a]\> quota
Disk quotas for user 689acd84ce6b11e28c1222000a8d945a (uid 507):
Filesystem blocks quota limit grace files quota limit grace
/dev/xvde2 1049016* 0 1048576 193 0 40000
|
Description of problem: rhc-admin-migrate will fail if the user has no disk space left. Version-Release number of selected component (if applicable): Latest How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Migrating gear on node with: /usr/bin/rhc-admin-migrate --login 'login' --migrate-gear '5134d41de0b8cd997d0003a2' --app-name 'v2' --version '2.0.28' Migrating on node... ***time_migrate_on_node_measured_from_broker=3311*** Migrate on node output: Beginning V1 -> V2 migration Inspecting gear at /var/lib/openshift/5134d41de0b8cd997d0003a2 Checking for V1 stop lock in ["php-5.3"] V1 stop lock not detected Marking step migrate_stop_lock complete Stopping gear with uuid '5134d41de0b8cd997d0003a2' on node 'ex-std-node125' Carts to migrate: {:framework_carts=>["php-5.3"], :leftover_carts=>["haproxy-1.4"]} Migrating cartridge php Shell command '/sbin/runuser -m -s /bin/sh 5134d41de0b8cd997d0003a2 -c "exec /usr/bin/runcon 'unconfined_u:system_r:openshift_t:s0:c1,c849' /bin/sh -c \"/var/lib/openshift/5134d41de0b8cd997d0003a2/ph p/bin/setup --version 5.3\""' returned an error. rc=1 stdout => stderr => cp: cannot create directory `/var/lib/openshift/5134d41de0b8cd997d0003a2/php//configuration/etc/conf': Disk quota exceeded cp: cannot create directory `/var/lib/openshift/5134d41de0b8cd997d0003a2/php//configuration/etc/conf.d': Disk quota exceeded cp: writing `/var/lib/openshift/5134d41de0b8cd997d0003a2/php//configuration/etc/php.ini.erb': Disk quota exceeded Migrate on node exit code: 1 from /usr/bin/rhc-admin-migrate:38:in `migrate_gear' from /usr/bin/rhc-admin-migrate:713:in `<main>' Expected results: It to migrate the gear Additional info: