Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1431971 - [regression] Unable to locate pgsql_data.tar & mongo_data.tar while compressing the katello-backup content
Summary: [regression] Unable to locate pgsql_data.tar & mongo_data.tar while compressi...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Backup & Restore
Version: 6.2.8
Hardware: All
OS: Linux
high
high
Target Milestone: Unspecified
Assignee: Christine Fouant
QA Contact: jcallaha
URL:
Whiteboard:
: 1433888 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-03-14 08:07 UTC by Rajan Gupta
Modified: 2020-07-16 09:18 UTC (History)
16 users (show)

Fixed In Version: katello-3.0.0-20
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1446720 (view as bug list)
Environment:
Last Closed: 2017-05-16 14:38:44 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
1431971.patch (1.08 KB, application/mbox)
2017-04-25 05:08 UTC, Mike McCune 		no flags Details
View All


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 17908 0 None None None 2017-03-15 21:25:25 UTC
Red Hat Product Errata RHBA-2017:1234 0 normal SHIPPED_LIVE Satellite Async Errata 2017-05-16 18:37:59 UTC

Description Rajan Gupta 2017-03-14 08:07:56 UTC 
Description of problem:
Unable to locate pgsql_data.tar & mongo_data.tar while compressing the katello-backup content


Version-Release number of selected component (if applicable):
6.2.8

How reproducible:
Execute "katello-backup" on satellite 6.2.8.

Steps to Reproduce:
1. Install/upgrade to satellite 6.2.8.
2. run "katello-backup <Path>


Actual results:
[root@sat6 /]# katello-backup --skip-pulp-content /katello-backup/
Starting backup: 2017-03-14 13:01:57 +0530
Creating backup folder /katello-backup/katello-backup-2017-03-14T13:01:57+05:30
Redirecting to /bin/systemctl stop  foreman-tasks.service
Redirecting to /bin/systemctl stop  httpd.service
Redirecting to /bin/systemctl stop  pulp_workers.service
Redirecting to /bin/systemctl stop  foreman-proxy.service
Redirecting to /bin/systemctl stop  pulp_streamer.service
Redirecting to /bin/systemctl stop  pulp_resource_manager.service
Redirecting to /bin/systemctl stop  pulp_celerybeat.service
Redirecting to /bin/systemctl stop  smart_proxy_dynflow_core.service
Redirecting to /bin/systemctl stop  tomcat.service
Redirecting to /bin/systemctl stop  squid.service
Redirecting to /bin/systemctl stop  qdrouterd.service
Redirecting to /bin/systemctl stop  qpidd.service
Redirecting to /bin/systemctl stop  postgresql.service
Redirecting to /bin/systemctl stop  mongod.service
Backing up config files... 
tar: Removing leading `/' from member names
Done.
Redirecting to /bin/systemctl stop  foreman-tasks.service
Redirecting to /bin/systemctl stop  httpd.service
Redirecting to /bin/systemctl stop  pulp_workers.service
Redirecting to /bin/systemctl stop  foreman-proxy.service
Redirecting to /bin/systemctl stop  pulp_streamer.service
Redirecting to /bin/systemctl stop  pulp_resource_manager.service
Redirecting to /bin/systemctl stop  pulp_celerybeat.service
Redirecting to /bin/systemctl stop  smart_proxy_dynflow_core.service
Redirecting to /bin/systemctl stop  tomcat.service
Redirecting to /bin/systemctl stop  squid.service
Redirecting to /bin/systemctl stop  qdrouterd.service
Redirecting to /bin/systemctl stop  qpidd.service
Redirecting to /bin/systemctl stop  postgresql.service
Redirecting to /bin/systemctl stop  mongod.service
Backing up postgres db... 
tar: Removing leading `/' from member names
Done.
Backing up mongo db... 
tar: Removing leading `/' from member names
Done.
Redirecting to /bin/systemctl start  mongod.service
Redirecting to /bin/systemctl start  postgresql.service
Redirecting to /bin/systemctl start  qpidd.service
Redirecting to /bin/systemctl start  qdrouterd.service
Redirecting to /bin/systemctl start  squid.service
Redirecting to /bin/systemctl start  tomcat.service
Redirecting to /bin/systemctl start  smart_proxy_dynflow_core.service
Redirecting to /bin/systemctl start  pulp_celerybeat.service
Redirecting to /bin/systemctl start  pulp_resource_manager.service
Redirecting to /bin/systemctl start  pulp_streamer.service
Redirecting to /bin/systemctl start  foreman-proxy.service
Redirecting to /bin/systemctl start  pulp_workers.service
Redirecting to /bin/systemctl start  httpd.service
Redirecting to /bin/systemctl start  foreman-tasks.service
gzip: pgsql_data.tar: No such file or directory
gzip: mongo_data.tar: No such file or directory
**** BACKUP Complete, contents can be found in: /katello-backup/katello-backup-2017-03-14T13:01:57+05:30 ****
[root@sat6 /]# 


Expected results:

[root@sat6 katello-backup]# katello-backup --skip-pulp-content /katello-backup/
Starting backup: 2017-03-14 13:15:45 +0530
Creating backup folder /katello-backup/katello-backup-2017-03-14T13:15:45+05:30
Redirecting to /bin/systemctl stop  foreman-tasks.service
Redirecting to /bin/systemctl stop  httpd.service
Redirecting to /bin/systemctl stop  pulp_workers.service
Redirecting to /bin/systemctl stop  foreman-proxy.service
Redirecting to /bin/systemctl stop  pulp_streamer.service
Redirecting to /bin/systemctl stop  pulp_resource_manager.service
Redirecting to /bin/systemctl stop  pulp_celerybeat.service
Redirecting to /bin/systemctl stop  smart_proxy_dynflow_core.service
Redirecting to /bin/systemctl stop  tomcat.service
Redirecting to /bin/systemctl stop  squid.service
Redirecting to /bin/systemctl stop  qdrouterd.service
Redirecting to /bin/systemctl stop  qpidd.service
Redirecting to /bin/systemctl stop  postgresql.service
Redirecting to /bin/systemctl stop  mongod.service
Backing up config files... 
tar: Removing leading `/' from member names
Done.
Redirecting to /bin/systemctl stop  foreman-tasks.service
Redirecting to /bin/systemctl stop  httpd.service
Redirecting to /bin/systemctl stop  pulp_workers.service
Redirecting to /bin/systemctl stop  foreman-proxy.service
Redirecting to /bin/systemctl stop  pulp_streamer.service
Redirecting to /bin/systemctl stop  pulp_resource_manager.service
Redirecting to /bin/systemctl stop  pulp_celerybeat.service
Redirecting to /bin/systemctl stop  smart_proxy_dynflow_core.service
Redirecting to /bin/systemctl stop  tomcat.service
Redirecting to /bin/systemctl stop  squid.service
Redirecting to /bin/systemctl stop  qdrouterd.service
Redirecting to /bin/systemctl stop  qpidd.service
Redirecting to /bin/systemctl stop  postgresql.service
Redirecting to /bin/systemctl stop  mongod.service
Backing up postgres db... 
tar: Removing leading `/' from member names
Done.
Backing up mongo db... 
tar: Removing leading `/' from member names
Done.
Redirecting to /bin/systemctl start  mongod.service
Redirecting to /bin/systemctl start  postgresql.service
Redirecting to /bin/systemctl start  qpidd.service
Redirecting to /bin/systemctl start  qdrouterd.service
Redirecting to /bin/systemctl start  squid.service
Redirecting to /bin/systemctl start  tomcat.service
Redirecting to /bin/systemctl start  smart_proxy_dynflow_core.service
Redirecting to /bin/systemctl start  pulp_celerybeat.service
Redirecting to /bin/systemctl start  pulp_resource_manager.service
Redirecting to /bin/systemctl start  pulp_streamer.service
Redirecting to /bin/systemctl start  foreman-proxy.service
Redirecting to /bin/systemctl start  pulp_workers.service
Redirecting to /bin/systemctl start  httpd.service
Redirecting to /bin/systemctl start  foreman-tasks.service
**** BACKUP Complete, contents can be found in: /katello-backup/katello-backup-2017-03-14T13:15:45+05:30 ****
[root@sat6 katello-backup]# 

Additional info:

While Analysing the "/usr/bin/katello-backup" script, I found below corrections:

1. Line number 68:
`tar --selinux --create --file=pgsql_data.tar.gz  --listed-incremental=.postgres.snar /var/lib/pgsql/data/`

change to >>>

`tar --selinux --create --file=pgsql_data.tar  --listed-incremental=.postgres.snar /var/lib/pgsql/data/`

AND

2. Line number 72: 
`tar --selinux --create --file=mongo_data.tar.gz --listed-incremental=.mongo.snar --exclude=mongod.lock /var/lib/mongodb/`

change to >>>

`tar --selinux --create --file=mongo_data.tar --listed-incremental=.mongo.snar --exclude=mongod.lock /var/lib/mongodb/`

This runs successfully.
Comment 6 Steven Ellis 2017-03-29 00:58:19 UTC 
There are two sides to the problem.

Backup script shouldn't complain about the missing tar files.

Recovery script should work correctly.
Comment 7 Muhammad Selim Jahangir 2017-04-12 23:13:21 UTC 
case#01718495,
Customer requested hotfix for satellite 6.2.8. Any update on this will be appreciated. 

Regards
Selim
Comment 9 Mike McCune 2017-04-25 05:07:18 UTC 
** HOTFIX For Satellite 6.2.8 ** 

1) download attached 1431971.patch to your Satellite

2) Copy to /usr/bin

3) patch -p2 < 1431971.patch

4) run katello-backup as normal, files should have the .tar.gz file extension and restore properly via katello-restore

If you upgrade to 6.2.9 please re-run the above hotfix patch to continue to carry this fix.
Comment 10 Mike McCune 2017-04-25 05:08:40 UTC 
Created attachment 1273813 [details]
1431971.patch
Comment 11 Xixi 2017-04-25 22:17:04 UTC 
(In reply to Mike McCune from comment #9)
> ** HOTFIX For Satellite 6.2.8 ** 
> 
Thank you - releasing hofix for Case 01718495 (Air New Zealand Limited, strategic)
Comment 12 Mike McCune 2017-04-27 14:38:17 UTC 
*** Bug 1433888 has been marked as a duplicate of this bug. ***
Comment 14 Rajan Gupta 2017-05-05 09:16:01 UTC 
This is partially fixed in 6.2.9

The online backup and restore now works
The offline backup is still creating tar files with a .gz extension and then trying to compress a non existant .tar file

gzip: pgsql_data.tar: No such file or directory
gzip: mongo_data.tar: No such file or directory

[root katello-backup-2017-05-05T14:49:08+08:00]# ls -la
total 34759604
drwxrwx--- 2 root postgres        4096 May  5 14:50 .
drwxr-xr-x 3 root root            4096 May  5 14:49 ..
-rw-r--r-- 1 root root        38785326 May  5 14:49 config_files.tar.gz
-rw-r--r-- 1 root root           95792 May  5 14:49 .config.snar
-rw-r--r-- 1 root root             346 May  5 14:53 metadata
-rw-r--r-- 1 root root     27720161280 May  5 14:53 mongo_data.tar.gz
-rw-r--r-- 1 root root             516 May  5 14:53 .mongo.snar
-rw-r--r-- 1 root root      7695124480 May  5 14:50 pgsql_data.tar.gz
-rw-r--r-- 1 root root           26159 May  5 14:50 .postgres.snar

[root katello-backup-2017-05-05T14:49:08+08:00]# file mongo_data.tar.gz
mongo_data.tar.gz: POSIX tar archive

[root katello-backup-2017-05-05T14:49:08+08:00]# file pgsql_data.tar.gz
pgsql_data.tar.gz: POSIX tar archive

See line 68 and 72 in /bin/katello-backup where it sets --file=pgsql_data.tar.gz, should be --file=pgsql_data.tar
and Line 83 and 84 where is tries to compress pgsql_data.tar"
Comment 15 jcallaha 2017-05-15 20:11:28 UTC 
Verified in Satellite 6.2.9 Async.

RHEL 7
------
-bash-4.2# katello-backup . --incremental katello-backup-2017-05-15T18\:05\:40+02\:00/
Starting backup: 2017-05-15 21:55:26 +0200
Creating backup folder ./katello-backup-2017-05-15T21:55:26+02:00
Generating metadata ... 
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_theme_satellite-0.1.42/app/models/concerns/satellite_packages.rb:4: warning: already initialized constant Katello::Ping::PACKAGES
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.125/app/models/katello/ping.rb:7: warning: previous definition of PACKAGES was here
Backing up config files... 
Done.
Redirecting to /bin/systemctl stop  foreman-tasks.service
Redirecting to /bin/systemctl stop  httpd.service
Redirecting to /bin/systemctl stop  pulp_workers.service
Redirecting to /bin/systemctl stop  foreman-proxy.service
Redirecting to /bin/systemctl stop  pulp_streamer.service
Redirecting to /bin/systemctl stop  pulp_resource_manager.service
Redirecting to /bin/systemctl stop  pulp_celerybeat.service
Redirecting to /bin/systemctl stop  smart_proxy_dynflow_core.service
Redirecting to /bin/systemctl stop  tomcat.service
Redirecting to /bin/systemctl stop  squid.service
Redirecting to /bin/systemctl stop  qdrouterd.service
Redirecting to /bin/systemctl stop  qpidd.service
Redirecting to /bin/systemctl stop  postgresql.service
Redirecting to /bin/systemctl stop  mongod.service
Backing up postgres db... 
tar: Removing leading `/' from member names
Done.
Backing up mongo db... 
tar: Removing leading `/' from member names
Done.
Backing up Pulp data... 
tar: Removing leading `/' from member names
Done.
Redirecting to /bin/systemctl start  mongod.service
Redirecting to /bin/systemctl start  postgresql.service
Redirecting to /bin/systemctl start  qpidd.service
Redirecting to /bin/systemctl start  qdrouterd.service
Redirecting to /bin/systemctl start  squid.service
Redirecting to /bin/systemctl start  tomcat.service
Redirecting to /bin/systemctl start  smart_proxy_dynflow_core.service
Redirecting to /bin/systemctl start  pulp_celerybeat.service
Redirecting to /bin/systemctl start  pulp_resource_manager.service
Redirecting to /bin/systemctl start  pulp_streamer.service
Redirecting to /bin/systemctl start  foreman-proxy.service
Redirecting to /bin/systemctl start  pulp_workers.service
Redirecting to /bin/systemctl start  httpd.service
Redirecting to /bin/systemctl start  foreman-tasks.service
Done with backup: 2017-05-15 21:57:45 +0200
**** BACKUP Complete, contents can be found in: ./katello-backup-2017-05-15T21:55:26+02:00 ****


RHEL 6
------

-bash-4.1# katello-backup .
Starting backup: Mon May 15 21:58:12 +0200 2017
Creating backup folder ./katello-backup-2017-05-15T21:58:12+02:00
Generating metadata ... 
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_theme_satellite-0.1.42/app/models/concerns/satellite_packages.rb:4: warning: already initialized constant Katello::Ping::PACKAGES
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.125/app/models/katello/ping.rb:7: warning: previous definition of PACKAGES was here
Backing up config files... 
Done.
celery multi v3.1.11 (Cipater)
> Stopping nodes...
	> reserved_resource_worker-6.eng.brq.redhat.com: QUIT -> 2335
	> reserved_resource_worker-1.eng.brq.redhat.com: QUIT -> 2217
	> reserved_resource_worker-5.eng.brq.redhat.com: QUIT -> 2312
	> reserved_resource_worker-7.eng.brq.redhat.com: QUIT -> 2358
	> reserved_resource_worker-0.eng.brq.redhat.com: QUIT -> 2201
	> reserved_resource_worker-2.eng.brq.redhat.com: QUIT -> 2242
	> reserved_resource_worker-4.eng.brq.redhat.com: QUIT -> 2289
	> reserved_resource_worker-3.eng.brq.redhat.com: QUIT -> 2266
> Waiting for 8 nodes -> 2335, 2217, 2312, 2358, 2201, 2242, 2289, 2266....................
	> reserved_resource_worker-6.eng.brq.redhat.com: OK
> Waiting for 7 nodes -> 2217, 2312, 2358, 2201, 2242, 2289, 2266....
	> reserved_resource_worker-1.eng.brq.redhat.com: OK
> Waiting for 6 nodes -> 2312, 2358, 2201, 2242, 2289, 2266....
	> reserved_resource_worker-5.eng.brq.redhat.com: OK
> Waiting for 5 nodes -> 2358, 2201, 2242, 2289, 2266....
	> reserved_resource_worker-7.eng.brq.redhat.com: OK
> Waiting for 4 nodes -> 2201, 2242, 2289, 2266....
	> reserved_resource_worker-0.eng.brq.redhat.com: OK
> Waiting for 3 nodes -> 2242, 2289, 2266....
	> reserved_resource_worker-2.eng.brq.redhat.com: OK
> Waiting for 2 nodes -> 2289, 2266....
	> reserved_resource_worker-4.eng.brq.redhat.com: OK
> Waiting for 1 node -> 2266....
	> reserved_resource_worker-3.eng.brq.redhat.com: OK

celery multi v3.1.11 (Cipater)
> Stopping nodes...
	> resource_manager.eng.brq.redhat.com: QUIT -> 1626
> Waiting for 1 node -> 1626......
	> resource_manager.eng.brq.redhat.com: OK

Backing up postgres db... 
tar: Removing leading `/' from member names
Done.
Backing up mongo db... 
tar: Removing leading `/' from member names
Done.
Backing up Pulp data... 
tar: Removing leading `/' from member names
Done.
celery multi v3.1.11 (Cipater)
> Starting nodes...
	> resource_manager.eng.brq.redhat.com: OK
celery multi v3.1.11 (Cipater)
> Starting nodes...
	> reserved_resource_worker-0.eng.brq.redhat.com: OK
	> reserved_resource_worker-1.eng.brq.redhat.com: OK
	> reserved_resource_worker-2.eng.brq.redhat.com: OK
	> reserved_resource_worker-3.eng.brq.redhat.com: OK
	> reserved_resource_worker-4.eng.brq.redhat.com: OK
	> reserved_resource_worker-5.eng.brq.redhat.com: OK
	> reserved_resource_worker-6.eng.brq.redhat.com: OK
	> reserved_resource_worker-7.eng.brq.redhat.com: OK
/usr/lib/python2.6/site-packages/pulp/server/db/connection.py:159: DeprecationWarning: add_son_manipulator is deprecated
  _DATABASE.add_son_manipulator(NamespaceInjector())
/usr/lib/python2.6/site-packages/pulp/server/db/model/base.py:96: DeprecationWarning: ensure_index is deprecated. Use create_index instead.
  unique=unique, background=True)
[Mon May 15 22:01:50 2017] [warn] module passenger_module is already loaded, skipping
Done with backup: Mon May 15 22:03:24 +0200 2017
**** BACKUP Complete, contents can be found in: ./katello-backup-2017-05-15T21:58:12+02:00 ****
-bash-4.1# ll
total 4.0K
drwxrwx---. 2 root postgres 4.0K May 15 22:03 katello-backup-2017-05-15T21:58:12+02:00
-bash-4.1# ll katello-backup-2017-05-15T21\:58\:12+02\:00/
total 507M
-rw-r--r--. 1 root root 610K May 15 21:58 config_files.tar.gz
-rw-r--r--. 1 root root  349 May 15 22:03 metadata
-rw-r--r--. 1 root root 401M May 15 22:00 mongo_data.tar.gz
-rw-r--r--. 1 root root  28M May 15 21:59 pgsql_data.tar.gz
-rw-r--r--. 1 root root  79M May 15 22:00 pulp_data.tar
Comment 17 errata-xmlrpc 2017-05-16 14:38:44 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1234
Note You need to log in before you can comment on or make changes to this bug.