Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1673797 - [RFE] Satellite backup/restore to have an option to skip /var/lib/qpidd and /var/lib/candlepin/activemq-artemis dirs
Summary: [RFE] Satellite backup/restore to have an option to skip /var/lib/qpidd and /...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Satellite Maintain
Version: 6.4
Hardware: All
OS: Linux
high
high
Target Milestone: Unspecified
Assignee: Suraj Patil
QA Contact: Lucie Vrtelova
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-02-08 06:46 UTC by Pavel Moravec
Modified: 2024-12-20 18:48 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-06-11 06:52:55 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)

Description Pavel Moravec 2019-02-08 06:46:53 UTC 
Description of problem:
foreman-maintain backup copies to the tarball also /var/lib/qpidd directory content. That is (in scaled environments) tens of GBs of usually unimportant data in empty pulp.agent.* queues. Example filesizes from one such environment:

1.2G	candlepin.dump
2.7G	config_files.tar.gz-without-qpidd
13G	config_files.tar.gz-orig-with-qpidd
1.3G	foreman.dump
4.0K	pg_globals.dump

Please add an option like skip pulp content to also skip qpidd content.

Pros:
- much smaller backup, skipping often redundant files
- much faster backup&restore times

Negs:
- katello-agent queues won't be backed up
- katello_event_queue won't be backed up (if not empty, katello won't know some subscription events)
- pulp tasks requests backlog lost
- during restore, extra need to create event exchange, katello_event_queue and 5 bindings between them

That can be important for reusing the backup at customer during disaster recovery, but usually it does not matter to keep that data when building internal reproducer else-where. For such purposes, the option --skip-qpidd-content would make sense.


Version-Release number of selected component (if applicable):
6.3, 6.4


How reproducible:
100%


Steps to Reproduce:
1. Have Satellite with many Hosts using katello-agent
2. foreman-maintain backup [online|offline] --skip-qpidd-content DIR
3. Check that /var/lib/qpidd/.qpidd/qls/jrnl2 is not collected in config_files.tar.gz in the backup.


Actual results:
2. no such option
3. the dir is present in either backup.


Expected results:
2. have such option to skip qpidd content


Additional info:
Comment 6 Pavel Moravec 2020-03-03 11:18:23 UTC 
We should also skip backing up:

/var/lib/candlepin/activemq-artemis

directory that spools event messages inside Artemis broker before they are sent to qpidd (plus it has two other purposes that are not important for Satellite). This directory content varies over time similarly like /var/lib/qpidd and tar-ing it can fail due to the same reason.
Comment 7 Pavel Moravec 2020-03-03 11:20:17 UTC 
(In reply to Pavel Moravec from comment #6)
> We should also skip backing up:
> 
> /var/lib/candlepin/activemq-artemis
> 
> directory that spools event messages inside Artemis broker before they are
> sent to qpidd (plus it has two other purposes that are not important for
> Satellite). This directory content varies over time similarly like
> /var/lib/qpidd and tar-ing it can fail due to the same reason.

But we still should backup:

/var/lib/candlepin/candlepin-crl.crl
/var/lib/candlepin/cpdb_done

I expect?
Comment 8 Pavel Moravec 2020-03-03 11:23:25 UTC 
(sorry for spamming)

nikos++ stated that since candlepin 3 / Sat7, the Artemis broker will be used for handling job framework, so it should be backed up, I expect. So only on Sat6, we can stop backing up /var/lib/candlepin/activemq-artemis dir
Comment 9 Sean O'Keeffe 2020-05-13 10:28:03 UTC 
If this is safe to skip, we should not present another option to the user for simplicity, and instead make this the default (maybe just for online backups?). If this is not safe to do, then we should close this - comment 0 leaves some doubt whether this is safe to do? "usually unimportant data"
Comment 10 Suraj Patil 2020-06-09 09:36:39 UTC 
Hello Pavel,

As the fix of below two bugs, now in online backup, we are not taking the backup of /var/lib/qpidd and /var/lib/candlepin/activemq-artemis. Also, in the restore process we create the required queues. So for an internal reproducer, we can take the online backup. 

Do you think that there is still need of --skip-qpidd-content option for offline backup? 


[1] https://bugzilla.redhat.com/show_bug.cgi?id=1673908
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1824134
Comment 11 Pavel Moravec 2020-06-11 06:52:55 UTC 
(In reply to Suraj Patil from comment #10)
> Hello Pavel,
> 
> As the fix of below two bugs, now in online backup, we are not taking the
> backup of /var/lib/qpidd and /var/lib/candlepin/activemq-artemis. Also, in
> the restore process we create the required queues. So for an internal
> reproducer, we can take the online backup. 
> 
> Do you think that there is still need of --skip-qpidd-content option for
> offline backup? 
> 
> 
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1673908
> [2] https://bugzilla.redhat.com/show_bug.cgi?id=1824134

Sounds enough, thanks. Closing this BZ as fixed in 6.7, then.
Note You need to log in before you can comment on or make changes to this bug.