Bug 1568443 - Incremental Backup is Capturing all Pulp Content
Summary: Incremental Backup is Capturing all Pulp Content
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Backup & Restore
Version: 6.3.1
Hardware: All
OS: Linux
unspecified
low
Target Milestone: Unspecified
Assignee: satellite6-bugs
QA Contact: Lucie Vrtelova
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-04-17 14:04 UTC by patalber
Modified: 2019-12-03 12:53 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-12-03 12:53:25 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description patalber 2018-04-17 14:04:04 UTC
Description of problem:
The command 'satellite-backup --incremental' results in a file that is similar in size to a full backup (depending on the timing of the backup). This is due to the entirety of the pulp content being backed up rather than on the new content since the full backup that is referenced in the command.

Version-Release number of selected component (if applicable):
katello-3.4.5-16.el7sat.noarch

How reproducible:
100%

Steps to Reproduce:
1. Take full backup: 
# sudo satellite-backup /satellitebackup --online-backup --snapshot --features all --assumeyes --logical-db-backup
2. Take an incremental backup (after synchronizing new packages): 
# sudo satellite-backup /satellitebackup --incremental /satellitebackup/satellite-backup-20180326134741 --snapshot --features all --assumeyes --logical-db-backup
3. Check the disk usage:
$ sudo du -sh *
276G    satellite-backup-20180326134741
276G    satellite-backup-20180328161035

Actual results:
The incremental backup is the same size as the full backup, taken after synchronizing new content.

Expected results:
The incremental backup is much smaller than the full backup, as only the new content is taken into account.

Additional info:

Comment 1 patalber 2018-04-17 14:06:45 UTC
Please ignore the '--online-backup' flag in the steps above.

Comment 3 Christine Fouant 2018-05-01 18:25:17 UTC
(In reply to patalber from comment #0)
> Description of problem:
> The command 'satellite-backup --incremental' results in a file that is
> similar in size to a full backup (depending on the timing of the backup).
> This is due to the entirety of the pulp content being backed up rather than
> on the new content since the full backup that is referenced in the command.
> 
> Version-Release number of selected component (if applicable):
> katello-3.4.5-16.el7sat.noarch
> 
> How reproducible:
> 100%
> 
> Steps to Reproduce:
> 1. Take full backup: 
> # sudo satellite-backup /satellitebackup --online-backup --snapshot
> --features all --assumeyes --logical-db-backup
> 2. Take an incremental backup (after synchronizing new packages): 
> # sudo satellite-backup /satellitebackup --incremental
> /satellitebackup/satellite-backup-20180326134741 --snapshot --features all
> --assumeyes --logical-db-backup
> 3. Check the disk usage:
> $ sudo du -sh *
> 276G    satellite-backup-20180326134741
> 276G    satellite-backup-20180328161035
> 
> Actual results:
> The incremental backup is the same size as the full backup, taken after
> synchronizing new content.
> 
> Expected results:
> The incremental backup is much smaller than the full backup, as only the new
> content is taken into account.
> 
> Additional info:

I believe this is actually due to https://bugzilla.redhat.com/show_bug.cgi?id=1573590 so not exactly a backup issue, but I'd like to verify the download policy on the repositories you are syncing are not 'on demand' as this will prevent the content from actually downloading.

I'm also wondering on some of the options you're running - Logical db backup and online backup are the same thing (a logical db backup is utilized as a failsafe if an offline backup is somehow corrupted). If you are using snapshots, it will always go offline momentarily, and then can perform a full backup while remaining online. So you're essentially running two online backups while also going offline while the snapshot is being created. Is this your intended usage?

Comment 4 patalber 2018-05-01 18:34:19 UTC
Hi Christine,

I will ask the client, as this is the command that he provided to me.

Thank you for the explanation.

--Patrick

Comment 5 patalber 2018-05-01 19:17:41 UTC
Christine,

Here is the client's response:

"The options we use are going to be up for discussion, if you can provide guidance.

Our ultimate goal is to have as much as possible backed up, with the least amount of downtime, though uptime is lower priority since we expect client systems to be using dedicated capsules, with the satellite server facilitating multiple capsules.  I was believing snapshotting and online backup would help towards that goal.

As for repo synchronization, it is my expectation that we always sync immediately.  We did play to ensure we're getting repos fully downloaded/propagated to capsules so as to reduce satellite server availability to client systems (we can allow for capsule downtime to be managed by specific customers, with only expected limitation being client registration changes, since we'd sync content during lifecycle environment promotions).
In webgui: settings: content: default repository download policy == immediate
In webgui: settings: content: default capsule download policy == immediate

I'm not finding where to ID this via hammer for repos other than default.  We do have nightly sync plan...
hammer> sync-plan info --id 1
ID:          1
Name:        Nightly Sync Plan
Start Date:  2018/03/07 07:00:00
Interval:    daily
Enabled:     yes
Description:
Created at:  2018/03/06 00:07:53
Updated at:  2018/03/06 00:07:53
Products:
 1) ID:   19
    Name: Red Hat Enterprise Linux Server
 2) ID:   27
    Name: Red Hat Satellite Capsule
 3) ID:   2
    Name: Red Hat Software Collections for RHEL Server
 4) ID:   45
    Name: Dell
 5) ID:   47
    Name: Red Hat Satellite
 6) ID:   53
    Name: EPEL
 7) ID:   11
    Name: Red Hat Enterprise Linux Server - Extended Update Support

hammer>"

Please let me know if you require more information.

Thank you.

--Patrick

Comment 7 Bryan Kearney 2019-11-04 14:33:55 UTC
The Satellite Team is attempting to provide an accurate backlog of bugzilla requests which we feel will be resolved in the next few releases. We do not believe this bugzilla will meet that criteria, and have plans to close it out in 1 month. This is not a reflection on the validity of the request, but a reflection of the many priorities for the product. If you have any concerns about this, feel free to contact Red Hat Technical Support or your account team. If we do not hear from you, we will close this bug out. Thank you.

Comment 8 Pat Riehecky 2019-11-05 14:15:17 UTC
Hello,

I believe the failure of incremental backups to backup incrementally is a serious bug.

Pat

Comment 9 Bryan Kearney 2019-12-03 12:53:25 UTC
Thank you for your interest in Satellite 6. We have evaluated this request, and while we recognize that it is a valid request, we do not expect this to be implemented in the product in the foreseeable future. This is due to other priorities for the product, and not a reflection on the request itself. We are therefore closing this out as WONTFIX. If you have any concerns about this, please do not reopen. Instead, feel free to contact Red Hat Technical Support. Thank you.


Note You need to log in before you can comment on or make changes to this bug.