Description of problem: Exporting RHEL 7 kickstart repository from a connected Satellite and trying to import it on a disconnected one fails with error "Error: ContentArtifact matching query does not exist." Version-Release number of selected component (if applicable): Satellite 6.10.1 How reproducible: Always, if the repository does not exists. Steps to Reproduce: 1. Sync the kickstart repository using immediate policy 2. Create a CV containing only this repository 3. Export it, using hammer 4. Copy the export directory the destination satellite (where the kickstart repository still doesn't exists) 5. Import it using hammer Actual results: Task goes to pause/error state and logs this traceback: error: traceback: |2 File "/usr/lib/python3.6/site-packages/pulpcore/tasking/pulpcore_worker.py", line 317, in _perform_task result = func(*args, **kwargs) File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 366, in publish publication_data.populate() File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 277, in populate setattr(self, f"{name}_repomdrecords", self.prepare_metadata_files(content, name)) File "/usr/lib/python3.6/site-packages/pulp_rpm/app/tasks/publishing.py", line 89, in prepare_metadata_files content_artifact = repo_metadata_file.contentartifact_set.get() File "/usr/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/usr/lib/python3.6/site-packages/django/db/models/query.py", line 408, in get self.model._meta.object_name description: ContentArtifact matching query does not exist. worker: "/pulp/api/v3/workers/111c5ae9-c32e-4c01-8f25-126f772c191d/" Expected results: Additional info:
(Just an update so folk know this is still being investigated) RHEL7-KS-REPO includes 3 productid files, one for the main repo and one for each of the addon-repos "inside" RHEL7-KS. AS an example from my test-machine, after syncing RHEL7 and RHEL7-KS here are the repometadata files: ===== pulp=> select rf.content_ptr_id, rf.relative_path, ca.pulp_id from rpm_repometadatafile rf LEFT JOIN core_contentartifact ca ON rf.content_ptr_id = ca.content_id; content_ptr_id | relative_path | pulp_id --------------------------------------+----------------------------------------------------------------------------------------+-------------------------------------- 430c399d-1381-476a-b162-fb3b10e8dfab | repodata/638a1ea9-66f5-4cd9-a1e8-505aa131e5b0 | 888a0329-7ce2-4eec-b5a7-2d91f6a56018 8b1e01b1-9557-4934-9cb4-bd9cd4b96fe1 | repodata/54cd0bb6d2ad360b27c710b0437a4396b0f69fdd610a3df70f37628eed8039b5-productid.gz | b8bbfba4-8e37-4dd9-8302-30a8364d5108 4a088a0b-c834-46b6-98ca-88f4b64ffd06 | repodata/716381b862bb7d32a7cc21e21e3c2ac167bd48332792da439b0806b97bd4cd92-productid.gz | 5212deb2-7d77-4954-9a00-5b2e24071245 c78d5dac-4b77-4852-8504-4516f0b577c5 | repodata/de7d562330fef4aaed524325e7f2050f54897ff5129d92245fca2f927b8ac319-productid.gz | ec85cb78-3589-41ea-b34a-628748f3d0f9 ===== and the repositories that hold them: ===== pulp=> select name, pulp_id from core_repository; name | pulp_id ------------------------------------------------------------------------------------------+-------------------------------------- rhel7 | 026383f2-da36-4960-876b-0e4928d25e07 rhel7_ks | 270b4eaf-8311-45df-83d7-bf94f2b413c1 addons/ResilientStorage-f8e0ccff1237e1d7638db88802f090aedfcad5379390cd21d4bbe5a03bc2448f | bfd3a593-1b2e-4a6e-ad7a-3b4d25b7f2ff addons/HighAvailability-f8e0ccff1237e1d7638db88802f090aedfcad5379390cd21d4bbe5a03bc2448f | 363996b5-bbfa-4e32-a879-bffc7d02dae2 (4 rows) ===== The two addons' productid-repometadata-files aren't ending up in the rhel7-ks export (and hence not in the import, leading to the traceback). The only ContentArtifact with productid in the rel-path in my export is this: ===== { "artifact": "54cd0bb6d2ad360b27c710b0437a4396b0f69fdd610a3df70f37628eed8039b5", "content": "8b1e01b1-9557-4934-9cb4-bd9cd4b96fe1", "relative_path": "repodata/54cd0bb6d2ad360b27c710b0437a4396b0f69fdd610a3df70f37628eed8039b5-productid.gz" }, ===== So - the problem is happening at export-time. Investigation continues.
Aaaand there it is. Consider this code: https://github.com/pulp/pulpcore/blob/main/pulpcore/app/modelresource.py#L94-L97 When exporting, you only get resources associated with the repository you are exporting. In a kickstart repository, addons are each considered **their own repository**. Therefore, their content is not exported (including these repometadata files). However, the **content** of those addon directories **are** considered part of RHEL7-KS - so the content, including repometadata content, gets exported. The ContentArtifact modelresource needs to know that some repositories "own" other repositories, and make sure to include such in the "parent" repo's export. Fun.
3rd para in #c3 was unclear/incorrect, trying again: When exporting, you only get resources associated with the repository you are exporting. In a kickstart repository, addons are each considered **their own repository**. Therefore, their ContentArtifact records ARE NOT exported. However, the **content** of those addon directories (including these repometadata files) **are** considered part of RHEL7-KS - so the content, including repometadata content, gets exported.
Fixed upstream, PR https://github.com/pulp/pulpcore/pull/2193 . Backported to core/3.14/15/16/17
Steps to retest 1. spin up 2 satellites: 1 for connected and another for disconnected 2. on connected, import a manifest 3. in settings, change the download policy to 'immediate' 4. sync a rhel7 ks 5. create a cv and add repo 6. publish cv 7. export cv 8. on disconnected satellite, import manifest 9. set the cdn to air-gapped 10. in settings, change the download policy to 'immediate' 11. run import expected: Import should pass successfully actual: import does pass successfully Verified on 6.11_017 with python38-pulpcore-3.16.7-1.el8pc.noarch. Tested on both rhel7 and rhel8
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: Satellite 6.11 Release), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:5498