Bug 1942745

Summary: pulp3: content migration fails for BaseOS and AppStream kickstart repos
Product: Red Hat Satellite Reporter: Tanya Tereshchenko <ttereshc>
Component: PulpAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED DUPLICATE QA Contact: Lai <ltran>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.9.0CC: ggainey, rchan, ttereshc
Target Milestone: 6.9.1Keywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-04-07 14:20:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Tanya Tereshchenko 2021-03-24 20:25:38 UTC
Description of problem:

If both CentOS8 BaseOS and Appstream kickstart repos are present in Pulp 2, it causes migration to fail.

It's not clear yet whether RHEL8 repos are affected. 


Steps to Reproduce:
1. Sync CentOS8 BaseOS and Appstream kickstart repos into pulp2
http://mirror.centos.org/centos/8/BaseOS/x86_64/os/
http://mirror.centos.org/centos/8/AppStream/x86_64/kickstart/
2. Run content migration

Actual results:

```
Mar  4 11:26:05 foreman pulpcore-worker-4: Traceback (most recent call last):
Mar  4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
Mar  4 11:26:05 foreman pulpcore-worker-4: rv = job.perform()
Mar  4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
Mar  4 11:26:05 foreman pulpcore-worker-4: self._result = self._execute()
Mar  4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
Mar  4 11:26:05 foreman pulpcore-worker-4: return self.func(*self.args, **self.kwargs)
Mar  4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py", line 81, in migrate_from_pulp2
Mar  4 11:26:05 foreman pulpcore-worker-4: migrate_content(plan, skip_corrupted=skip_corrupted)
Mar  4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 47, in migrate_content
Mar  4 11:26:05 foreman pulpcore-worker-4: plugin.migrator.migrate_content_to_pulp3(skip_corrupted=skip_corrupted)
Mar  4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/migrator.py", line 145, in migrate_content_to_pulp3
Mar  4 11:26:05 foreman pulpcore-worker-4: loop.run_until_complete(dm.create())
Mar  4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
Mar  4 11:26:05 foreman pulpcore-worker-4: return future.result()
Mar  4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 90, in create
Mar  4 11:26:05 foreman pulpcore-worker-4: await pipeline
Mar  4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
Mar  4 11:26:05 foreman pulpcore-worker-4: await asyncio.gather(*futures)
Mar  4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
Mar  4 11:26:05 foreman pulpcore-worker-4: await self.run()
Mar  4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 244, in run
Mar  4 11:26:05 foreman pulpcore-worker-4: RemoteArtifact.objects.bulk_get_or_create(self._needed_remote_artifacts(batch))
Mar  4 11:26:05 foreman pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 293, in _needed_remote_artifacts
Mar  4 11:26:05 foreman pulpcore-worker-4: msg.format(rp=content_artifact.relative_path, c=d_content.content)
Mar  4 11:26:05 foreman pulpcore-worker-4: ValueError: No declared artifact with relative path "images/boot.iso" for content "<DistributionTree: pk=6995fa03-af4e-4f86-b324-a9888309a5e7>"
```

Expected results:

No failure and successful migration

Comment 2 Tanya Tereshchenko 2021-04-07 14:20:10 UTC
The fix will be backported upstream to the pulpcore 3.7.z as a part of BZ#1945377

Comment 3 Tanya Tereshchenko 2021-04-07 14:20:46 UTC

*** This bug has been marked as a duplicate of bug 1945377 ***

Comment 4 pulp-infra@redhat.com 2021-04-07 15:12:02 UTC
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.

Comment 5 pulp-infra@redhat.com 2021-04-07 15:12:03 UTC
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 6 pulp-infra@redhat.com 2021-04-08 21:10:37 UTC
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.