Red Hat Bugzilla – Bug 996491
Do not check quota explicitly when "snapshot" an app with large database which consumes 90% quota
Last modified: 2015-05-14 19:26:02 EDT
Created attachment 786095 [details]
an example to create large data file
Description of problem:
When trying to snapshot save an application with a big database files ( see it has consumed more than 900M) , it reports success even if met disk quota exceed error. but actually it did not save the whole database dump successfully after confirming it by restore the saved snapshot back to another app.
Did not find any explict errors or warnings in in /var/log/mcollective.log either even if the client message is blocked by bug https://bugzilla.redhat.com/show_bug.cgi?id=994174
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. Create an app with database added, e.g. rhc app create php1 php-5.3 mysql-5.1
2. SSH into the app. Fill the app with database data. An example could be sql script attached to the bug. Make sure that after this step, more than 90% space is used.
3. On instance, tailf ./var/log/mcollective.log
4. rhc snapshot save php1
5. Create another app.
rhc app create php2 php-5.3 mysql-5.1
6. Restore the file generated from the first app php1 to php2.
rhc snapshot restore -a php2 -f php1.tag.gz
[root@ip-10-164-47-156 ~]# rhc snapshot save php1
Pulling down a snapshot to php1.tar.gz...
gzip: stdout: Disk quota exceeded
mysqldump: Got errno 32 on write
Couldn't not dump mysql! Continuing anyway
Creating and sending tar.gz
1). It shows "Disk quota exceeded" although it ends up "success".
2). If restoring this archive to other app (e.g php2 with mysql), the database will not be restored.
3). No logs is added to /var/log/mcollective.log.
Quota check should be done when snapshot an app. If the quota is used more than 90%(or other percentage we want to inform the user), a warning message should be given.
Commit pushed to master at https://github.com/openshift/origin-server
Bug 996491: Show warning when using snapshot and near disk quota
When the gear user's disk usage is >= 90% of quota, display a warning when
snapshotting or restoring (as these operations can fail in unexpected ways
when disk writes fail).
Tested on devenv_3683.
When "snapshot save" an app with more than 90% storage used, I got the following warning message:
"WARNING: The application's disk usage is very close to the quota limit. The snapshot may fail unexpectedly"
When "restore", I got a warning like:
"WARNING: The application's disk usage is very close to the quota limit. The restore may fail unexpectedly"
Mark the bug as verified.