Bug 1958415
| Summary: | On_demand capsule sync (pulp3 to pulp2) for non-sha256 repos fails with a checksum type issue | ||
|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Tanya Tereshchenko <ttereshc> |
| Component: | Pulp | Assignee: | satellite6-bugs <satellite6-bugs> |
| Status: | CLOSED ERRATA | QA Contact: | Vladimír Sedmík <vsedmik> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 6.9.0 | CC: | egolov, ggainey, rchan, ttereshc, vsedmik |
| Target Milestone: | 6.10.0 | Keywords: | Triaged |
| Target Release: | Unused | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | python-pulp-rpm-3.11.0 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-11-16 14:10:56 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
(In reply to Tanya Tereshchenko from comment #0) > > A workaround is to perform a capsule sync with on_demand policy. A workaround is to perform a capsule sync with the *immediate* policy. The Pulp upstream bug status is at NEW. Updating the external tracker on this bug. The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug. The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug. The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug. The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug. The Pulp upstream bug status is at POST. Updating the external tracker on this bug. The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug. All upstream Pulp bugs are at MODIFIED+. Moving this bug to POST. This issue is only partially fixed and can be fully fixed only in the later versions (the ones which use pulp_rpm 3.13+). The majority of repositories use sha256 checksum type, so this issue affects the small amount of repositories. There are 2 workarounds available: - use immediate policy with capsule sync - on the main Satellite explicitly specify a checksum type which needs to be used for a repo (if remote source uses md5, specify md5 to be used; sha1 -> sha1, etc) The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug. The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug. Verified with: 1) Satellite 6.10.0 snap 9 (upgraded from 6.9.4 snap 1), python3-pulpcore-3.14.0-1.el7pc.noarch 2) Capsule 6.9.4 snap 1, pulp-server-2.21.5.2-1.el7sat.noarch Sync of an on-demand sha512 repo to the on-demand Capsule succeeded, no errors seen in the journal, repo created in capsule:/var/lib/pulp The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug. The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: Satellite 6.10 Release), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:4702 |
Description of problem: pulp_rpm 3.10 always publishes metadata with sha256 checksum type(in repomd.xml). pulp 2 decides based on metadata type, which checksum type to expect for content as well (in primary.xml). If one syncs a repo with md5 checksumtype (for content) using on_emand policy, sha256 checksum won't be available, and this is where pulp2 on_demand sync fails. Both content migration and sync into main Satellite are affected. pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) Exception caught from plugin during publish for repo [rpm-with-md5] pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) Traceback (most recent call last): pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 1303, in _do_publish pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) publish_report = publish_repo(transfer_repo, conduit, call_config) pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 905, in wrap_f pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) return f(*args, **kwargs) pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 185, in publish_repo pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) return self._publisher.process_lifecycle() pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 573, in process_lifecycle pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) super(PluginStep, self).process_lifecycle() pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 164, in process_lifecycle pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) step.process() pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 240, in process pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) self._process_block(item=item) pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 302, in _process_block pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) self.process_main(item=item) pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 500, in process_main pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) context.add_unit_metadata(unit) pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/metadata/filelists.py", line 42, in add_unit_metadata pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) self.metadata_file_handle.write(unit.render_filelists(self.checksum_type)) pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/db/models.py", line 868, in render_filelists pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) context = Context({'pkgid': self.get_or_calculate_and_save_checksum(checksumtype)}) pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/db/models.py", line 258, in get_or_calculate_and_save_checksum pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) checksumtype=checksumtype) pulp.server.controllers.repository:ERROR: [1bbde036] (8758-47744) PulpCodedException: Checksum type "sha256" is not available for all units in the repository. Make sure those units have been downloaded. Steps to Reproduce: - Sync using on_demand policy a non-sha256 repo into a main Satellite (Pulp 3). - Trigger capsule (Pulp 2) sync, also on_demand(this is important) of a repo from step 1. A workaround is to perform a capsule sync with on_demand policy.