Bug 1944733 - [RFE] There is no option to do an incremental content view export using 'hammer content-view version export'
Summary: [RFE] There is no option to do an incremental content view export using 'hamm...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Inter Satellite Sync
Version: 6.8.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: 6.10.0
Assignee: satellite6-bugs
QA Contact: Lai
URL:
Whiteboard:
: 1679736 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-03-30 15:07 UTC by Jayant Bhatia
Modified: 2023-09-07 23:19 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-11-16 14:10:29 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
Import CV (53.13 KB, image/png)
2021-06-23 19:22 UTC, Lai
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 6207202 0 None None None 2021-07-22 03:44:24 UTC
Red Hat Product Errata RHSA-2021:4702 0 None None None 2021-11-16 14:10:42 UTC

Description Jayant Bhatia 2021-03-30 15:07:13 UTC
Description of problem:


Version-Release number of selected component (if applicable): All Satellite versions


How reproducible: Consider a scenario where there is an offline and online Satellite. The content view needs to be exported from an online Satellite and needs to be imported to an offline Satellite (disconnected Satellite).


Steps to Reproduce:
1. Running 'hammer content-view version export' for the first time export all the data/content from content view.

2. Next time (say after a month), if I need to export content view again using 'hammer content-view version export' option, there is no option to only export the updated / changed packages on subsequent exports.

Actual results:
Running 'hammer content-view version export' always export the complete content view.

Expected results:
There should be an option to export only the changed/updated content. Something similar to "--since SINCE" option present with " hammer content-view version export-legacy" command.

Additional info:

Comment 2 Brad Buckingham 2021-05-26 18:45:07 UTC
With Satellite 6.10, the support for performing an incremental export can now be supported.

Below are a couple of example commands involving either a content view version or Library:

hammer content-export incremental version --content-view=view --version=2.0 --organization="Default_Organization"

hammer content-export incremental library --organization="Default_Organization"

Comment 3 Lai 2021-06-23 19:21:54 UTC
Note: The testing of hammer content-import version --organization-id <org-id> --path /var/lib/pulp/imports/<dir with import> can be found here: https://bugzilla.redhat.com/show_bug.cgi?id=1632961 and that has been successful.  This testing will focus on "hammer content-export incremental library"


Steps to test:

1. Spin up 2 Satellites (6.10 each)

#Export Satellite Portion

2. On satellite to export, import manifest
3. Administer -> Setting -> Search for "Default Red Hat Repository download policy" -> Set policy to "immediate"
4. Enable a several RH repos
5. Sync the repos
6. Run complete export command: hammer content-export complete library --organization-id <org-id>
7. Verify that the export files are created in "/var/lib/pulp/exports/<organization name>/Export-Library/<version>/<export time stamp>
8. Check on UI to see that a "Export-Library" cv is created which contains all repos from step 4

#Import Disconnected Satellite Portion

9. On the satellite to import the cv, import manifest that would have all the repos created in step 4
10. Set "Default Red Hat Repository download policy" to "immediate" just like step 3.
11. Set "Disconnected Mode" to "Yes" by going to Administer -> Setting -> Content Tab -> Disconnected Mode
12. SCP the files from the exported directory in step 8 to /var/lib/pulp/imports in the satellite in which you want to import
13. Provide write permission to the dir imported: chown -R pulp:pulp /var/lib/pulp/imports/<dir with import>
14. Verify that the files are present in /var/lib/pulp/imports/<dir with import>
15. Run the import command: hammer content-import library --organization-id <org-id> --path /var/lib/pulp/imports/<dir with import>
16. Verify that on the UI that a CV named "Import-Library" is created and contains all the repos from step 4

# Back to Export Satellite portion to test incremental export

18. Enable and sync a new repo
19. Run incremental export command: hammer content-export incremental library --organization-id <org-id>
20. Check the repos within the metadata.json file by running: cat <dir of exported content>|json_reformat
21. Compare the sizes of the tar.gz files between the complete and incremental version: ls -lh <dir of exported content>
22. Check UI to see "Export-Library" cv has version 2.0 with the new repos included

# Back to Import Disconnected Satellite Portion

23. SCP the files from the incremental exported directory in step 19 to /var/lib/pulp/imports in the satellite in which you want to import
24. Provide write permission to the dir imported: chown -R pulp:pulp /var/lib/pulp/imports/<dir with incremental import>
25. Verify that the files are present in /var/lib/pulp/imports/<dir with import>
26. Run the import command: hammer content-import version --organization-id <org-id> --path /var/lib/pulp/imports/<dir with import>
27. Verify that on the UI that the CV and it's content is all imported over for "Import-Library" for version 2.0 

Expected:
20) Metadata should have all repos and ["incremental": true,]
21) Incremental file should be smaller than complete file
22) UI should have "Export-Library" with version 2.0 with new repo included
26) Import should be successful
27) UI should show "Import-Library" with version 2.0 with new repo included

Actual:
20) Metadata does have all repos and
 "incremental": true,
21)
[]# ls -lh /var/lib/pulp/exports/Default_Organization/Export-Library/1.0/2021-06-23T14-37-04-04-00/  <-----Complete Export
total 1.4G
-rw-r--r--. 1 pulp pulp 1.4G Jun 23 14:38 export-b57cd967-070e-4877-9008-2407c1b77112-20210623_1837.tar.gz
-rw-r--r--. 1 pulp pulp  333 Jun 23 14:38 export-b57cd967-070e-4877-9008-2407c1b77112-20210623_1837-toc.json
-rw-r--r--. 1 root root 5.0K Jun 23 14:38 metadata.json
[]# ls -lh /var/lib/pulp/exports/Default_Organization/Export-Library/2.0/2021-06-23T14-48-15-04-00/  <-----Incremental Export
total 5.6M
-rw-r--r--. 1 pulp pulp 5.5M Jun 23 14:48 export-ff7b7982-9876-441b-ab4e-184670c1061c-20210623_1848.tar.gz
-rw-r--r--. 1 pulp pulp  333 Jun 23 14:48 export-ff7b7982-9876-441b-ab4e-184670c1061c-20210623_1848-toc.json
-rw-r--r--. 1 root root 6.0K Jun 23 14:48 metadata.json
22) UI does have "Export-Library" with version 2.0 and new repo included
26) Import is successfull
27) UI does have "Import-Library" with version 2.0 and new repo included

Verified on 6.10_snap 5

Comment 4 Lai 2021-06-23 19:22:39 UTC
Created attachment 1793620 [details]
Import CV

Comment 5 Brad Buckingham 2021-07-16 14:59:30 UTC
*** Bug 1679736 has been marked as a duplicate of this bug. ***

Comment 8 errata-xmlrpc 2021-11-16 14:10:29 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Satellite 6.10 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:4702


Note You need to log in before you can comment on or make changes to this bug.