Bug 996491 - Do not check quota explicitly when "snapshot" an app with large database which consumes 90% quota
Do not check quota explicitly when "snapshot" an app with large database whi...
Status: CLOSED CURRENTRELEASE
Product: OpenShift Online
Classification: Red Hat
Component: Containers (Show other bugs)
2.x
Unspecified Unspecified
medium Severity medium
: ---
: ---
Assigned To: Dan Mace
libra bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-08-13 05:41 EDT by Qiushui Zhang
Modified: 2015-05-14 19:26 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-08-29 08:50:35 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
an example to create large data file (1.78 KB, text/plain)
2013-08-13 05:41 EDT, Qiushui Zhang
no flags Details

  None (edit)
Description Qiushui Zhang 2013-08-13 05:41:10 EDT
Created attachment 786095 [details]
an example to create large data file

Description of problem:
When trying to snapshot save an application with a big database files ( see it has consumed more than 900M) , it reports success  even if met disk quota  exceed error. but actually it did not save the whole database dump successfully after confirming it by restore the saved snapshot  back to another app.
Did not find any explict errors or warnings in  in /var/log/mcollective.log  either even if  the client message is blocked by bug https://bugzilla.redhat.com/show_bug.cgi?id=994174
Version-Release number of selected component (if applicable):
devenv_3640

How reproducible:
always

Steps to Reproduce:
1. Create an app with database added, e.g. rhc app create php1 php-5.3 mysql-5.1
2. SSH into the app. Fill the app with database data. An example could be sql script attached to the bug. Make sure that after this step, more than 90% space is used.
3. On instance, tailf ./var/log/mcollective.log
4. rhc snapshot save php1
5. Create another app. 
   rhc app create php2 php-5.3 mysql-5.1
6. Restore the file generated from the first app php1 to php2. 
   rhc snapshot restore -a php2 -f php1.tag.gz

Actual results:
[root@ip-10-164-47-156 ~]# rhc snapshot save php1
Pulling down a snapshot to php1.tar.gz...
gzip: stdout: Disk quota exceeded
mysqldump: Got errno 32 on write
Couldn't not dump mysql! Continuing anyway
Creating and sending tar.gz
RESULT:
Success
1). It shows "Disk quota exceeded" although it ends up "success".
2). If restoring this archive to other app (e.g php2 with mysql), the database will not be restored.
3). No logs is added to /var/log/mcollective.log.


Expected results:
Quota check should be done when snapshot an app. If the quota is used more than 90%(or other percentage we want to inform the user), a warning message should be given.


Additional info:
Comment 2 openshift-github-bot 2013-08-21 14:01:34 EDT
Commit pushed to master at https://github.com/openshift/origin-server

https://github.com/openshift/origin-server/commit/a55be7528dd3063aad988cb7c864a56a9164b34e
Bug 996491: Show warning when using snapshot and near disk quota

When the gear user's disk usage is >= 90% of quota, display a warning when
snapshotting or restoring (as these operations can fail in unexpected ways
when disk writes fail).
Comment 3 Qiushui Zhang 2013-08-21 22:34:39 EDT
Tested on devenv_3683.
When "snapshot save" an app with more than 90% storage used, I got the following warning message:
"WARNING: The application's disk usage is very close to the quota limit. The snapshot may fail unexpectedly"


When "restore", I got a warning like:
"WARNING: The application's disk usage is very close to the quota limit. The restore may fail unexpectedly"

Mark the bug as verified.

Note You need to log in before you can comment on or make changes to this bug.