Bug 1659888 - Not a good idea to store undercloud backups in Swift
Summary: Not a good idea to store undercloud backups in Swift
Keywords:
Status: NEW
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-tripleoclient
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: ---
: ---
Assignee: Juan Badia Payno
QA Contact: Gurenko Alex
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-12-17 06:33 UTC by Dan Macpherson
Modified: 2020-02-11 05:04 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Target Upstream Version:


Attachments (Terms of Use)

Description Dan Macpherson 2018-12-17 06:33:20 UTC
When running the "openstack undercloud backup" command, tripleo creates an archive containing the full undercloud backup and the undercloud filesystem and saves it as an object in swift.

The problem is the filesystem backup also includes the swift filesystem, which will include the previous backups. This means that any sequential backup you take will exponentially increase in size.

For example, here are two backups I took the same day:

(undercloud) [stack@ccsosp-undercloud ~]$ ls -lh UC*
-rw-rw-r--. 1 stack stack 6.6G Dec 17 15:53 UC-backup-20181217154339.tar
-rw-rw-r--. 1 stack stack  14G Dec 17 16:13 UC-backup-20181217155754.tar

The second backup jumps in size because it also contains the first backup as a swift object.

The next backup I take will be 6.6GB (inital filesystem) + 6.6.GB (backup 1) + 14GB (backup 2) ~= 28GB

The next backup after that will be 6.6GB + 6.6.GB + 14GB + 28GB ~= 56GB

etc, etc, etc

There are probably a few other reasons not to store the archive in Swift:

* If the undercloud disk gets corrupted and causes the undercloud to become inaccessible, you won't be able to recreate the undercloud since the backup is stored in the undercloud.
* The undercloud has a recommended disk size of 100GB, which can fill up quickly if performing a daily backup.

Might be an idea to simply export the archive as a file so the user can move it to an external location.

Comment 1 Adriano Petrich 2018-12-18 13:49:42 UTC
I feel that this is more related to user action than to the swift system. Probably changing the default would be enough. What would be the default action?

The exponential backup problem seems bad but it seems like the correct behaviour if you want a backup of your backup these things happen.

Comment 2 Dan Macpherson 2018-12-18 14:44:48 UTC
(In reply to Adriano Petrich from comment #1)
> I feel that this is more related to user action than to the swift system.
> Probably changing the default would be enough. What would be the default
> action?

When you say "default action", are we talking about mistral actions? If so, the relevant actions are:

* tripleo.undercloud.create_file_system_backup, which maps to class CreateFileSystemBackup in tripleo-common -- this creates the filesystem backup, including the swift objects
* tripleo.undercloud.upload_backup_to_swift which maps to class UploadUndercloudBackupToSwift in tripleo-common -- this saves the resulting backup to swift

So might need to change the component of this BZ to tripleo-common

> The exponential backup problem seems bad but it seems like the correct
> behaviour if you want a backup of your backup these things happen.

It seems a little redundant and impractical to backup a backup and then upload it to the same location as the original backup. I'm not sure why anyone would want this to be standard behaviour.

Comment 3 Adriano Petrich 2019-07-23 14:30:29 UTC
Agreed in all counts.

When I said the default action it was bad semantics. I meant should the default be store to disk? because that is already an intermediary step. What could be done is that uploading to swift and deleting the temporary directory be an optional step. The new backups would not contain the previous ones if they are not in swift and the backup dir is outside the regular folders to be backupped.

Comment 4 Dan Macpherson 2019-07-23 14:38:21 UTC
Pretty much. Ideally you'd want store the backup in a location not included in the filesystem backup to avoid the exponential backup size.

Comment 6 Juan Badia Payno 2019-11-27 16:10:51 UTC
We are moving all the backup and restore feature to use ReaR.


Note You need to log in before you can comment on or make changes to this bug.