Bug 1964530

Summary:	[pulp3] ValueError: No declared artifact with relative path ".treeinfo" for migrating CentOS 8 kickstart repos
Product:	Red Hat Satellite	Reporter:	Tanya Tereshchenko <ttereshc>
Component:	Pulp	Assignee:	satellite6-bugs <satellite6-bugs>
Status:	CLOSED ERRATA	QA Contact:	Lai <ltran>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	6.9.0	CC:	egolov, ggainey, jjeffers, rchan, ttereshc
Target Milestone:	6.9.4	Keywords:	Triaged
Target Release:	Unused
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	python-pulp_2to3_migration-0.11.2	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1973358 (view as bug list)		Environment:
Last Closed:	2021-07-29 12:58:50 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Tanya Tereshchenko 2021-05-25 16:08:01 UTC

Description of problem:

In some cases the error is raised for CentOS 8 kickstart repos migration.

CentOS 8 kickstart repos are created in an unusual way.
Usual way: One distribution tree repo with variants and addons.
Unusual way (CentOS8 repos only so far): N repos with the same name and release distinguishable only by the build_timestamp.

The issue depends on the order of migration, so not every setup will hit it.

Apr 21 07:36:18 stg-katello pulpcore-worker-3: Traceback (most recent call last):
Apr 21 07:36:18 stg-katello pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
Apr 21 07:36:18 stg-katello pulpcore-worker-3: rv = job.perform()
Apr 21 07:36:18 stg-katello pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
Apr 21 07:36:18 stg-katello pulpcore-worker-3: self._result = self._execute()
Apr 21 07:36:18 stg-katello pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
Apr 21 07:36:18 stg-katello pulpcore-worker-3: return self.func(*self.args, **self.kwargs)
Apr 21 07:36:18 stg-katello pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py", line 81, in migrate_from_pulp2
Apr 21 07:36:18 stg-katello pulpcore-worker-3: migrate_content(plan, skip_corrupted=skip_corrupted)
Apr 21 07:36:18 stg-katello pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 47, in migrate_content
Apr 21 07:36:18 stg-katello pulpcore-worker-3: plugin.migrator.migrate_content_to_pulp3(skip_corrupted=skip_corrupted)
Apr 21 07:36:18 stg-katello pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/rpm/migrator.py", line 145, in migrate_content_to_pulp3
Apr 21 07:36:18 stg-katello pulpcore-worker-3: loop.run_until_complete(dm.create())
Apr 21 07:36:18 stg-katello pulpcore-worker-3: File "/usr/lib64/python3.6/asyncio/base_events.py", line 484, in run_until_complete
Apr 21 07:36:18 stg-katello pulpcore-worker-3: return future.result()
Apr 21 07:36:18 stg-katello pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/plugin/content.py", line 89, in create
Apr 21 07:36:18 stg-katello pulpcore-worker-3: await pipeline
Apr 21 07:36:18 stg-katello pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 225, in create_pipeline
Apr 21 07:36:18 stg-katello pulpcore-worker-3: await asyncio.gather(*futures)
Apr 21 07:36:18 stg-katello pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/api.py", line 43, in __call__
Apr 21 07:36:18 stg-katello pulpcore-worker-3: await self.run()
Apr 21 07:36:18 stg-katello pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 244, in run
Apr 21 07:36:18 stg-katello pulpcore-worker-3: RemoteArtifact.objects.bulk_get_or_create(self._needed_remote_artifacts(batch))
Apr 21 07:36:18 stg-katello pulpcore-worker-3: File "/usr/lib/python3.6/site-packages/pulpcore/plugin/stages/artifact_stages.py", line 301, in _needed_remote_artifacts
Apr 21 07:36:18 stg-katello pulpcore-worker-3: msg.format(rp=content_artifact.relative_path, c=d_content.content)
Apr 21 07:36:18 stg-katello pulpcore-worker-3: ValueError: No declared artifact with relative path ".treeinfo" for content "<DistributionTree: pk=d0a0b30f-8cfc-4d76-96da-c2317f720819>"


Steps to reproduce:

1. Sync into pulp2 and then migrate CentOS 8 Appstream or PowerTools or HighAvailability (not Extras and not BaseOS).
2. Then sync into pulp2 CentOS 8 BaseOS and migrate, observe the error. BaseOS should be migrated the last to reproduce the issue.

Comment 1 pulp-infra@redhat.com 2021-05-25 16:28:25 UTC

The Pulp upstream bug status is at POST. Updating the external tracker on this bug.

Comment 2 pulp-infra@redhat.com 2021-05-25 16:28:27 UTC

The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 3 pulp-infra@redhat.com 2021-05-25 18:50:38 UTC

The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.

Comment 4 pulp-infra@redhat.com 2021-05-25 19:29:54 UTC

All upstream Pulp bugs are at MODIFIED+. Moving this bug to POST.

Comment 6 pulp-infra@redhat.com 2021-06-14 15:08:50 UTC

The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.

Comment 7 Lai 2021-07-07 18:28:54 UTC

Steps to test

1. Create a product with the following custom repos: centos8 appstream, centos08 powertool, centos08 highavailability
2. Sync repos
3. Perform migration
4. Check results

Expected:
Repos should migrate successfull

Actual:
Repos migrated successfully without issues.


Verified on 6.9.4_01 with python3-pulp-2to3-migration-0.11.2-1.el7pc.noarch

Comment 8 Lai 2021-07-07 18:29:54 UTC

Also checked on pulp-cli and repos are present there.

Comment 13 errata-xmlrpc 2021-07-29 12:58:50 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Satellite 6.9.4 Async Bug Fix Update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:2948

Comment 14 errata-xmlrpc 2021-07-29 13:02:56 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Satellite 6.9.4 Async Bug Fix Update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:2948