Bug 1673797

Summary:	[RFE] Satellite backup/restore to have an option to skip /var/lib/qpidd and /var/lib/candlepin/activemq-artemis dirs
Product:	Red Hat Satellite	Reporter:	Pavel Moravec <pmoravec>
Component:	Satellite Maintain	Assignee:	Suraj Patil <supatil>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Lucie Vrtelova <lvrtelov>
Severity:	high	Docs Contact:
Priority:	high
Version:	6.4	CC:	apatel, bkearney, dgross, inecas, jpathan, kgaikwad, mbacovsk, momran, patalber, sokeeffe, sthirugn, supatil
Target Milestone:	Unspecified	Keywords:	FutureFeature, PrioBumpField, PrioBumpGSS, Triaged
Target Release:	Unused
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-06-11 06:52:55 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Pavel Moravec 2019-02-08 06:46:53 UTC

Description of problem:
foreman-maintain backup copies to the tarball also /var/lib/qpidd directory content. That is (in scaled environments) tens of GBs of usually unimportant data in empty pulp.agent.* queues. Example filesizes from one such environment:

1.2G candlepin.dump
2.7G config_files.tar.gz-without-qpidd
13G config_files.tar.gz-orig-with-qpidd
1.3G foreman.dump
4.0K pg_globals.dump

Please add an option like skip pulp content to also skip qpidd content.

Pros:
- much smaller backup, skipping often redundant files
- much faster backup&restore times

Negs:
- katello-agent queues won't be backed up
- katello_event_queue won't be backed up (if not empty, katello won't know some subscription events)
- pulp tasks requests backlog lost
- during restore, extra need to create event exchange, katello_event_queue and 5 bindings between them

That can be important for reusing the backup at customer during disaster recovery, but usually it does not matter to keep that data when building internal reproducer else-where. For such purposes, the option --skip-qpidd-content would make sense.

Version-Release number of selected component (if applicable):
6.3, 6.4

How reproducible:
100%

Steps to Reproduce:
1. Have Satellite with many Hosts using katello-agent
2. foreman-maintain backup [online|offline] --skip-qpidd-content DIR
3. Check that /var/lib/qpidd/.qpidd/qls/jrnl2 is not collected in config_files.tar.gz in the backup.

Actual results:
2. no such option
3. the dir is present in either backup.

Expected results:
2. have such option to skip qpidd content

Additional info:

Comment 6 Pavel Moravec 2020-03-03 11:18:23 UTC

We should also skip backing up:

/var/lib/candlepin/activemq-artemis

directory that spools event messages inside Artemis broker before they are sent to qpidd (plus it has two other purposes that are not important for Satellite). This directory content varies over time similarly like /var/lib/qpidd and tar-ing it can fail due to the same reason.

Comment 7 Pavel Moravec 2020-03-03 11:20:17 UTC

(In reply to Pavel Moravec from comment #6)
> We should also skip backing up:
> 
> /var/lib/candlepin/activemq-artemis
> 
> directory that spools event messages inside Artemis broker before they are
> sent to qpidd (plus it has two other purposes that are not important for
> Satellite). This directory content varies over time similarly like
> /var/lib/qpidd and tar-ing it can fail due to the same reason.

But we still should backup:

/var/lib/candlepin/candlepin-crl.crl
/var/lib/candlepin/cpdb_done

I expect?

Comment 8 Pavel Moravec 2020-03-03 11:23:25 UTC

(sorry for spamming)

nikos++ stated that since candlepin 3 / Sat7, the Artemis broker will be used for handling job framework, so it should be backed up, I expect. So only on Sat6, we can stop backing up /var/lib/candlepin/activemq-artemis dir

Comment 9 Sean O'Keeffe 2020-05-13 10:28:03 UTC

If this is safe to skip, we should not present another option to the user for simplicity, and instead make this the default (maybe just for online backups?). If this is not safe to do, then we should close this - comment 0 leaves some doubt whether this is safe to do? "usually unimportant data"

Comment 10 Suraj Patil 2020-06-09 09:36:39 UTC

Hello Pavel,

As the fix of below two bugs, now in online backup, we are not taking the backup of /var/lib/qpidd and /var/lib/candlepin/activemq-artemis. Also, in the restore process we create the required queues. So for an internal reproducer, we can take the online backup. 

Do you think that there is still need of --skip-qpidd-content option for offline backup? 


[1] https://bugzilla.redhat.com/show_bug.cgi?id=1673908
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1824134

Comment 11 Pavel Moravec 2020-06-11 06:52:55 UTC

(In reply to Suraj Patil from comment #10)
> Hello Pavel,
> 
> As the fix of below two bugs, now in online backup, we are not taking the
> backup of /var/lib/qpidd and /var/lib/candlepin/activemq-artemis. Also, in
> the restore process we create the required queues. So for an internal
> reproducer, we can take the online backup. 
> 
> Do you think that there is still need of --skip-qpidd-content option for
> offline backup? 
> 
> 
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1673908
> [2] https://bugzilla.redhat.com/show_bug.cgi?id=1824134

Sounds enough, thanks. Closing this BZ as fixed in 6.7, then.