Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2164413 - backup restore unable to cope with backups created via "-t" option
Summary: backup restore unable to cope with backups created via "-t" option
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Satellite Maintain
Version: 6.12.0
Hardware: x86_64
OS: Linux
high
high
Target Milestone: 6.13.0
Assignee: Pavel Moravec
QA Contact: Jameer Pathan
URL:
Whiteboard:
: 2121082 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-01-25 13:51 UTC by Pavel Moravec
Modified: 2023-06-20 17:09 UTC (History)
5 users (show)

Fixed In Version: rubygem-foreman_maintain-1.2.5
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-05-03 13:24:53 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 36012 0 Normal Ready For Testing backup restore unable to cope with backups created via "-t" option 2023-02-14 13:47:08 UTC
Red Hat Issue Tracker SAT-15196 0 None None None 2023-01-26 15:13:18 UTC
Red Hat Knowledge Base (Solution) 6995556 0 None None None 2023-01-26 14:35:16 UTC
Red Hat Product Errata RHSA-2023:2097 0 None None None 2023-05-03 13:25:00 UTC

Description Pavel Moravec 2023-01-25 13:51:29 UTC
Description of problem:
When a backup is created with option "-t" (to split pulp_data.tar into smaller chunks), satellite-maintain restore is *completely* unable to restore that backup.

The cause is, -t option implies tar uses --tape-length option which implies -M option. To skip prompts, --new-volume-script=/usr/share/gems/gems/foreman_maintain-1.1.8/bin/foreman-maintain-rotate-tar mimics exchanging tapes.

That works great on backup, where pulp_data.part* files are created.

But restore routine does not use that at all, causing the tar --extract gets stuck forever.

It "just" needs to add:

https://github.com/theforeman/foreman_maintain/blob/9d5f5c9315e71d08451d51fb2d41dc0f735bdfc9/definitions/features/tar.rb#L56

to the tar --extract routine (every time? or just when backup contains some pulp_data.part* file?)


I can verify manually that just adding that argument to a "split backup" makes the tar to successfully complete..


Version-Release number of selected component (if applicable):
Sat 6.11 +


How reproducible:
100%


Steps to Reproduce:
1. create a backup split to smaller chunks:

mkdir /backup
satellite-maintain backup offline -t 50M --whitelist="sync-plans-enable" /backup

2. try to restore it:

satellite-maintain restore --whitelist="sync-plans-enable" -y /backup/satellite-backup-2023-01-25-14-19-21

3. Wait until "Extracting pulp data" occurs for a while.


Actual results:
3. When "Extracting pulp data" occurs, see that "ps aux | grep tar" shows some tar command being idle. B'cos it waits on user prompt to swap the tapes :(

And the "Extracting pulp data" never completes.


Expected results:
3. "Extracting pulp data" to complete in reasonable time.


Additional info:

Comment 1 Pavel Moravec 2023-01-25 15:43:35 UTC
Some additional notes:

satellite-maintain restore procedure does not have "-t" option (which adds the --new-volume-script option when creating the archive, and which misses during extract).


Potential workaround: 
1) run the command:

tar --selinux --extract --file=/path/to/satellite-backup-2023-01-23-14-17-07/pulp_data.tar --absolute-names --overwrite --listed-incremental=/dev/null -M --directory=/ --new-volume-script=/usr/share/gems/gems/foreman_maintain-1.1.8/bin/foreman-maintain-rotate-tar

(add the latest argument there)

2) rename the pulp_data.tar
3) create a dummy pulp_data.tar, like:

touch /tmp/empty
tar cf /PATH/TO/BACKUP/pulp_data.tar /tmp/empty

4) Let the restore run again - it will not stuck on the same user prompt, but extract the dummy file instead; pulp data extracted manually in the right way

(procedure is pending a review, use with caution until KCS is created)

Comment 2 Pavel Moravec 2023-01-25 21:44:51 UTC
Perspective patch:

--- /usr/share/gems/gems/foreman_maintain-1.1.8/definitions/features/tar.rb.orig	2023-01-25 20:40:20.781292213 +0100
+++ /usr/share/gems/gems/foreman_maintain-1.1.8/definitions/features/tar.rb.new	2023-01-25 20:39:40.311747413 +0100
@@ -51,13 +51,16 @@ class Features::Tar < ForemanMaintain::F
     end
 
     if volume_size
-      split_tar_script = default_split_tar_script
       tar_command << "--tape-length=#{volume_size}"
-      tar_command << "--new-volume-script=#{split_tar_script}"
     end
 
     tar_command << '--overwrite' if options[:overwrite]
-    tar_command << '--gzip' if options[:gzip]
+
+    if options[:gzip]
+      tar_command << '--gzip'
+    else
+      tar_command << '-M --new-volume-script=#{default_split_tar_script}'
+    end
 
     exclude = options.fetch(:exclude, [])
     exclude.each do |ex|

Comment 3 Pavel Moravec 2023-01-25 22:25:23 UTC
(In reply to Pavel Moravec from comment #2)
> +      tar_command << '-M --new-volume-script=#{default_split_tar_script}'

It must be:

+      tar_command << "-M --new-volume-script=#{default_split_tar_script}"

to eval the variable properly.

Comment 4 Jean-Baptiste Dancre 2023-01-26 08:33:22 UTC
One would need to confirm, but i'm not 100% sure the -M is needed when you pass the --new-volume-script flag.

In theory, extract from manpages of tar:
~~~
-F, --info-script=NAME, --new-volume-script=NAME
run script at end of each tape (implies -M)
~~~

Comment 5 Pavel Moravec 2023-01-26 14:04:31 UTC
I can confirm that the patch from #c2 + #c3 does work well on any backup, either with -t or without the option used. Also, it properly applies the -M --new-volume-script options only to pulp_data.tar - not to gzipped tarbals that cant be split that way.

And indeed, we can even omit the -M option there, since it is implicit from the other one. So the working patch is:

--- /usr/share/gems/gems/foreman_maintain-1.1.8/definitions/features/tar.rb.orig	2023-01-25 20:40:20.781292213 +0100
+++ /usr/share/gems/gems/foreman_maintain-1.1.8/definitions/features/tar.rb.new	2023-01-25 20:39:40.311747413 +0100
@@ -51,13 +51,16 @@ class Features::Tar < ForemanMaintain::F
     end
 
     if volume_size
-      split_tar_script = default_split_tar_script
       tar_command << "--tape-length=#{volume_size}"
-      tar_command << "--new-volume-script=#{split_tar_script}"
     end
 
     tar_command << '--overwrite' if options[:overwrite]
-    tar_command << '--gzip' if options[:gzip]
+
+    if options[:gzip]
+      tar_command << '--gzip'
+    else
+      tar_command << "--new-volume-script=#{default_split_tar_script}"
+    end
 
     exclude = options.fetch(:exclude, [])
     exclude.each do |ex|


It is not very clean patch as we should detect the tape size in a more clever way, but.. at least it works. As I successfully tested all combinations "backup|restore satellite with|without -t option".

Comment 6 Bryan Kearney 2023-01-27 00:03:34 UTC
Upstream bug assigned to pmoravec

Comment 7 Bryan Kearney 2023-01-27 00:03:36 UTC
Upstream bug assigned to pmoravec

Comment 8 Eric Helms 2023-02-09 13:55:05 UTC
*** Bug 2121082 has been marked as a duplicate of this bug. ***

Comment 9 Bryan Kearney 2023-02-16 20:03:28 UTC
Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/36012 has been resolved.

Comment 10 Jameer Pathan 2023-03-20 16:49:04 UTC
Verified

Verified with:
- Satellite 6.13.0 snap 15
- rubygem-foreman_maintain-1.2.7-1

Test steps:
- mkdir /backup
- satellite-maintain backup offline -t 50M /backup
- satellite-maintain restore -y /backup/satellite-backup-2023-03-20-09-59-46/

Observation:
- No error during restore.

Comment 13 errata-xmlrpc 2023-05-03 13:24:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.13 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:2097


Note You need to log in before you can comment on or make changes to this bug.