Bug 1445989 - Incremental backup script in effect is performing full backup.
Summary: Incremental backup script in effect is performing full backup.
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Backup & Restore
Version: 6.2.8
Hardware: Unspecified
OS: Unspecified
high vote
Target Milestone: Unspecified
Assignee: Christine Fouant
QA Contact: jcallaha
: 1446683 (view as bug list)
Depends On:
Blocks: 1446070
TreeView+ depends on / blocked
Reported: 2017-04-26 23:16 UTC by Tapomoy Bhowmik
Modified: 2021-08-30 13:28 UTC (History)
6 users (show)

Fixed In Version: katello-3.0.0-20
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2017-05-16 14:39:12 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 19446 0 High Closed Incremental backup script in effect is performing full backup. 2021-01-28 11:15:02 UTC
Red Hat Knowledge Base (Solution) 3015091 0 None None None 2018-12-06 20:45:54 UTC
Red Hat Knowledge Base (Solution) 3091601 0 None None None 2017-06-23 02:48:07 UTC
Red Hat Product Errata RHBA-2017:1234 0 normal SHIPPED_LIVE Satellite Async Errata 2017-05-16 18:37:59 UTC

Description Tapomoy Bhowmik 2017-04-26 23:16:51 UTC
Description of problem: Incremental backup script in effect is performing full backup.

One error similar to the following:

cp: cannot stat ‘*.snar’: No such file or directory

It seems that the below files are used to provide a diff.  As they weren't being read, customer was getting a full backup instead of an incremental backup. 


Adding the '.' to the command on line 131 fixed the problem:

`cp #{@options[:incremental]}/.*.snar .` if @options[:incremental]

Below is the difference between two files.

# diff -u katello-backup katello-backup.new 
--- katello-backup	2017-02-03 10:01:31.000000000 -0500
+++ katello-backup.new	2017-04-25 09:56:54.162505948 -0400
@@ -128,7 +128,7 @@
   `tar --selinux --create --gzip --file=config_files.tar.gz --listed-incremental=.config.snar #{CONFIGS.join(' ')}`
   puts "Done."
-  `cp #{@options[:incremental]}/*.snar .` if @options[:incremental]
+  `cp #{@options[:incremental]}/.*.snar .` if @options[:incremental]
   if @options[:online]

Version-Release number of selected component (if applicable):

How reproducible: Reproducible at customer's environment.

Steps to Reproduce: Take incremental backup using the below command.

# katello-backup /path/to/dir --incremental [PREVIOUS_BACKUP_DIR]

Actual results: Incremental backup script is performing full backup.

Expected results: It should perform incremental backup.

Additional info: Customer had provided the above solution which they expect to be included in future release or to fix the issue.

Comment 1 Amit Kumar Das 2017-04-27 00:16:23 UTC
Confirming that the issue exist in sat 6.2.8.

After following documentation here:

However, every time we perform an incremental backup it appears to do a full backup. The size of the backup is the same as the full backup. The incremental and full backups complete successfully but take a long time and consume a large amount of disk due to the incremental being full backups. 

Backup commands used are:
# katello-backup /var/backup/
# katello-backup /var/backup/test --incremental /var/backup
#katello-backup /var/backup/test --incremental /var/backup/katello-backup-2017-04-25T15\:26\:32+08\:00

Reproducing results from local satellite server 

# pwd
# katello-backup /var/backup/
# ls -lrt
drwxrwx---. 2 root postgres 183 Apr 25 15:10 katello-backup-2017-04-25T15:09:42+08:00
# du -sh katello-backup-2017-04-25T15:09:42+08:00
4.9G	katello-backup-2017-04-25T15:09:42+08:00
# ls -lrt katello-backup-2017-04-25T15:09:42+08:00
total 5097380
-rw-r--r--. 1 root root     628681 Apr 25 15:10 config_files.tar.gz
-rw-r--r--. 1 root root  239749120 Apr 25 15:10 pgsql_data.tar.gz
-rw-r--r--. 1 root root 4327495680 Apr 25 15:10 mongo_data.tar.gz
-rw-r--r--. 1 root root  651837440 Apr 25 15:10 pulp_data.tar
# cd katello-backup-2017-04-25T15\:09\:42+08\:00/
# du -sh *
616K	config_files.tar.gz
4.1G	mongo_data.tar.gz
229M	pgsql_data.tar.gz
622M	pulp_data.tar


# pwd
# katello-backup /var/backup/test --incremental /var/backup
# ls -lrt
drwxrwx---. 2 root postgres 183 Apr 25 15:27 katello-backup-2017-04-25T15:26:32+08:00
# du -sh katello-backup-2017-04-25T15\:26\:32+08\:00/
5.0G	katello-backup-2017-04-25T15:26:32+08:00/
# ls -lrt katello-backup-2017-04-25T15\:26\:32+08\:00/
total 5177452
-rw-r--r--. 1 root root     627702 Apr 25 15:26 config_files.tar.gz
-rw-r--r--. 1 root root  241244160 Apr 25 15:26 pgsql_data.tar.gz
-rw-r--r--. 1 root root 4327495680 Apr 25 15:27 mongo_data.tar.gz
-rw-r--r--. 1 root root  732334080 Apr 25 15:27 pulp_data.tar
# du -sh *
616K	config_files.tar.gz
4.1G	mongo_data.tar.gz
231M	pgsql_data.tar.gz
699M	pulp_data.tar

Comment 4 Christine Fouant 2017-05-03 13:39:43 UTC
*** Bug 1446683 has been marked as a duplicate of this bug. ***

Comment 5 Evgeni Golov 2017-05-03 14:01:28 UTC
Created redmine issue http://projects.theforeman.org/issues/19446 from this bug

Comment 7 Satellite Program 2017-05-12 02:05:44 UTC
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/19446 has been resolved.

Comment 8 jcallaha 2017-05-15 20:59:00 UTC
Verified in Satellite 6.2.9 Async.

The backup script is now properly limiting incremental backups.

-bash-4.2# katello-backup . --incremental katello-backup-2017-05-15T18\:05\:40+02\:00/
Starting backup: 2017-05-15 21:55:26 +0200
Creating backup folder ./katello-backup-2017-05-15T21:55:26+02:00
Generating metadata ... 
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_theme_satellite-0.1.42/app/models/concerns/satellite_packages.rb:4: warning: already initialized constant Katello::Ping::PACKAGES
/opt/theforeman/tfm/root/usr/share/gems/gems/katello- warning: previous definition of PACKAGES was here
Backing up config files... 
Redirecting to /bin/systemctl stop  foreman-tasks.service
Redirecting to /bin/systemctl stop  httpd.service
Redirecting to /bin/systemctl stop  pulp_workers.service
Redirecting to /bin/systemctl stop  foreman-proxy.service
Redirecting to /bin/systemctl stop  pulp_streamer.service
Redirecting to /bin/systemctl stop  pulp_resource_manager.service
Redirecting to /bin/systemctl stop  pulp_celerybeat.service
Redirecting to /bin/systemctl stop  smart_proxy_dynflow_core.service
Redirecting to /bin/systemctl stop  tomcat.service
Redirecting to /bin/systemctl stop  squid.service
Redirecting to /bin/systemctl stop  qdrouterd.service
Redirecting to /bin/systemctl stop  qpidd.service
Redirecting to /bin/systemctl stop  postgresql.service
Redirecting to /bin/systemctl stop  mongod.service
Backing up postgres db... 
tar: Removing leading `/' from member names
Backing up mongo db... 
tar: Removing leading `/' from member names
Backing up Pulp data... 
tar: Removing leading `/' from member names
Redirecting to /bin/systemctl start  mongod.service
Redirecting to /bin/systemctl start  postgresql.service
Redirecting to /bin/systemctl start  qpidd.service
Redirecting to /bin/systemctl start  qdrouterd.service
Redirecting to /bin/systemctl start  squid.service
Redirecting to /bin/systemctl start  tomcat.service
Redirecting to /bin/systemctl start  smart_proxy_dynflow_core.service
Redirecting to /bin/systemctl start  pulp_celerybeat.service
Redirecting to /bin/systemctl start  pulp_resource_manager.service
Redirecting to /bin/systemctl start  pulp_streamer.service
Redirecting to /bin/systemctl start  foreman-proxy.service
Redirecting to /bin/systemctl start  pulp_workers.service
Redirecting to /bin/systemctl start  httpd.service
Redirecting to /bin/systemctl start  foreman-tasks.service
Done with backup: 2017-05-15 21:57:45 +0200
**** BACKUP Complete, contents can be found in: ./katello-backup-2017-05-15T21:55:26+02:00 ****
-bash-4.2# ll katello-backup-2017-05-15T
ls: cannot access katello-backup-2017-05-15T: No such file or directory
-bash-4.2# ll
total 0
drwxrwx---. 2 root postgres 199 May 15 18:06 katello-backup-2017-05-15T18:05:40+02:00
drwxrwx---. 2 root postgres 199 May 15 21:57 katello-backup-2017-05-15T21:55:26+02:00
-bash-4.2# ll katello-backup-2017-05-15T21\:55\:26+02\:00/
total 656M
-rw-r--r--. 1 root root 605K May 15 21:55 config_files.tar.gz
-rw-r--r--. 1 root root  346 May 15 21:57 metadata
-rw-r--r--. 1 root root 181M May 15 21:56 mongo_data.tar.gz
-rw-r--r--. 1 root root  27M May 15 21:56 pgsql_data.tar.gz
-rw-r--r--. 1 root root 448M May 15 21:56 pulp_data.tar
-bash-4.2# space
1.3G	./katello-backup-2017-05-15T18:05:40+02:00
659M	./katello-backup-2017-05-15T21:55:26+02:00
1.9G	.

Comment 10 errata-xmlrpc 2017-05-16 14:39:12 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.