Bug 1950511
| Summary: | [pulp3] ISO content migration fails with ValueError: Cannot create repository version. Path is duplicated: <filename> | ||
|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Tanya Tereshchenko <ttereshc> |
| Component: | Pulp | Assignee: | satellite6-bugs <satellite6-bugs> |
| Status: | CLOSED ERRATA | QA Contact: | Lai <ltran> |
| Severity: | medium | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 6.9.0 | CC: | ggainey, jjeffers, rchan, ttereshc, zhunting |
| Target Milestone: | 6.9.2 | Keywords: | Triaged |
| Target Release: | Unused | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | pulp-2to3-migration-0.11.1 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-05-20 18:05:30 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
The Pulp upstream bug status is at NEW. Updating the external tracker on this bug. The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug. It can only happen on a system upgraded from Sat < 6.3. The pulp bug https://pulp.plan.io/issues/2773 which allowed files with the same name to be associated with a repo was fixed in pulp 2.13.3 (Sat 6.3). Updates to the steps to reproduce. > To reproduce: > - create a pulp2 repo with 2 different files (checksum differs) which will have the same filename. in Sat 6.2. - upgrade to Sat 6.9 > - migrate such repo to pulp 3 The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug. The Pulp upstream bug status is at POST. Updating the external tracker on this bug. The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug. All upstream Pulp bugs are at MODIFIED+. Moving this bug to POST. The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug. Steps to retest. Note: This was retested using dogfood proper which was configured to only migrate file ISOs. Duplicated paths have been confirmed by devs, but with the fix, the errors are not shown anymore for "ValueError: Cannot create repository version. Path is duplicated: <filename>" after migration. The proper steps to retest if need: 1. Get dogfood proper 2. Have dev configure to only migrate ISOs and restart dynflow 3. Run migration Expected: Migration should complete without the error "ValueError: Cannot create repository version. Path is duplicated: <filename>". Actual: Migration completed without the error "ValueError: Cannot create repository version. Path is duplicated: <filename>". Verified on 6.9.2 snap 3 with python3-pulp-2to3-migration-0.11.1-1.el7pc.noarch. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Satellite 6.9.2 Async Bug Fix Update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:2074 |
Description of problem: In Pulp 2, it seems to be possible to add N different files with the same filename (relative path) to the same repo. Only one is published but according to the database there are N files with the same name in one repo. For that reason, migration plugin tries to create a repo version with all N files with the same name. Here is the valid failure but the migration needs to handle it: ``` pulp: rq.worker:ERROR: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 124, in validate_version_paths validate_file_paths(paths) File "/usr/lib/python3.6/site-packages/pulpcore/app/files.py", line 133, in validate_file_paths raise ValueError(_("Path is duplicated: {path}").format(path=path)) ValueError: Path is duplicated: katello-installer-3.0.0.88.tar.gz During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job rv = job.perform() File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform self._result = self._execute() File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute return self.func(*self.args, **self.kwargs) File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 194, in complex_repo_migration create_repo_version(progress_rv, pulp2_repo, pulp3_remote) File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/migration.py", line 344, in create_repo_version new_version.remove_content(Content.objects.filter(pk__in=to_delete)) File "/usr/lib/python3.6/site-packages/pulpcore/app/models/repository.py", line 795, in __exit__ repository.finalize_new_version(self) File "/usr/lib/python3.6/site-packages/pulp_file/app/models.py", line 79, in finalize_new_version validate_repo_version(new_version) File "/usr/lib/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 139, in validate_repo_version validate_version_paths(version) File "/usr/lib/python3.6/site-packages/pulpcore/plugin/repo_version_utils.py", line 126, in validate_version_paths raise ValueError(_("Cannot create repository version. {err}.").format(err=e)) ValueError: Cannot create repository version. Path is duplicated: katello-installer-3.0.0.88.tar.gz. ``` To reproduce: - create a pulp2 repo with 2 different files (checksum differs) which will have the same filename. - migrate such repo to pulp 3