Description of problem: There are cases when migration or re-migration may fail at the stage of processing artifacts. Those are usually hard to reproduce, they often depend on the specific data or on the combination of content ended up migrating in one batch (which is quite random). The cases below are reproducible to the extent. If content from a repo with an importer in pulp 2 is copied to a new repo without a remote and then published and it all happens between migration re-runs, one might see the following error. See also the steps provided by a user https://pulp.plan.io/issues/7876#note-5 Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: Traceback (most recent call last): Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: rv = job.perform() Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: self._result = self._execute() Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: return self.func(*self.args, **self.kwargs) Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py", line 139, in migrate_from_pulp2 Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: migrate_content(plan) Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 36, in migrate_content Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: plugin.migrator.migrate_content_to_pulp3() Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/migrator.py", line 140, in migrate_content_to_pulp3 Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: loop.run_until_complete(dm.create()) Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: return future.result() Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 88, in create Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: await pipeline Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: await asyncio.gather(*futures) Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__ Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: await self.run() Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 244, in run Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: RemoteArtifact.objects.bulk_get_or_create(self._needed_remote_artifacts(batch)) Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 296, in _needed_remote_artifacts Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: if remote_artifact.remote_id == d_artifact.remote.pk: Nov 19 19:04:08 lxserv2285.smhi.se rq[1208174]: AttributeError: 'NoneType' object has no attribute 'pk' Another case: sync these specific repos http://mirror.centos.org/centos/8/BaseOS/x86_64/os/.treeinfo http://mirror.centos.org/centos/8/AppStream/x86_64/kickstart/.treeinfo and migrate them, and you might get the following error. Mar 4 11:26:05 foreman pulpcore-worker-4: Traceback (most recent call last): Mar 4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job Mar 4 11:26:05 foreman pulpcore-worker-4: rv = job.perform() Mar 4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform Mar 4 11:26:05 foreman pulpcore-worker-4: self._result = self._execute() Mar 4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute Mar 4 11:26:05 foreman pulpcore-worker-4: return self.func(*self.args, **self.kwargs) Mar 4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py", line 81, in migrate_from_pulp2 Mar 4 11:26:05 foreman pulpcore-worker-4: migrate_content(plan, skip_corrupted=skip_corrupted) Mar 4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 47, in migrate_content Mar 4 11:26:05 foreman pulpcore-worker-4: plugin.migrator.migrate_content_to_pulp3(skip_corrupted=skip_corrupted) Mar 4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/migrator.py", line 145, in migrate_content_to_pulp3 Mar 4 11:26:05 foreman pulpcore-worker-4: loop.run_until_complete(dm.create()) Mar 4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete Mar 4 11:26:05 foreman pulpcore-worker-4: return future.result() Mar 4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 90, in create Mar 4 11:26:05 foreman pulpcore-worker-4: await pipeline Mar 4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline Mar 4 11:26:05 foreman pulpcore-worker-4: await asyncio.gather(*futures) Mar 4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__ Mar 4 11:26:05 foreman pulpcore-worker-4: await self.run() Mar 4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 244, in run Mar 4 11:26:05 foreman pulpcore-worker-4: RemoteArtifact.objects.bulk_get_or_create(self._needed_remote_artifacts(batch)) Mar 4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 293, in _needed_remote_artifacts Mar 4 11:26:05 foreman pulpcore-worker-4: msg.format(rp=content_artifact.relative_path, c=d_content.content) Mar 4 11:26:05 foreman pulpcore-worker-4: ValueError: No declared artifact with relative path "images/boot.iso" for content "<DistributionTree: pk=6995fa03-af4e-4f86-b324-a9888309a5e7>"
The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.
*** Bug 1946366 has been marked as a duplicate of this bug. ***
*** Bug 1942745 has been marked as a duplicate of this bug. ***
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.
All upstream Pulp bugs are at MODIFIED+. Moving this bug to POST.
I checked the commit code and compared it to the one on a box with the latest build. Looks like all the changes are there. Verified on 6.9.1_02 with python3-pulpcore-3.7.5-1.el7pc.noarch
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Satellite 6.9.1 Async Bug Fix Update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:1371