Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1821182 - [Repository] - Packages are not getting synced with rpm-with-sha-512
Summary: [Repository] - Packages are not getting synced with rpm-with-sha-512
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Pulp
Version: 6.7.0
Hardware: All
OS: All
unspecified
high
Target Milestone: 6.8.0
Assignee: satellite6-bugs
QA Contact: Lai
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-04-06 09:10 UTC by Omkar Khatavkar
Modified: 2023-10-06 19:35 UTC (History)
15 users (show)

Fixed In Version: pulp-rpm-2.21.2
Doc Type: Known Issue
Doc Text:
Using the "Immediate" download policy to synchronize RPM repositories that are signed with SHA-512 hashes causes the synchronization to fail with the following error: "KeyError: u'sha512'". To work around this issue, change the download policy to "On Demand" for repositories that are signed with SHA-512 hashes.
Clone Of:
Environment:
Last Closed: 2020-10-27 13:01:16 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Pulp Redmine 6453 0 Normal CLOSED - CURRENTRELEASE sha512 checksum validation fail 2020-05-19 16:32:37 UTC
Red Hat Knowledge Base (Solution) 5167961 0 None None None 2020-06-18 15:39:07 UTC
Red Hat Product Errata RHSA-2020:4366 0 None None None 2020-10-27 13:01:47 UTC

Description Omkar Khatavkar 2020-04-06 09:10:38 UTC 
Description of problem:
[Repository] - Packages are not getting copied over with rpm-with-sha-512

Version-Release number of selected component (if applicable):
exist in 6.7 snap 20 
Not exist in 6.6  

How reproducible:
always

Steps to Reproduce:
1.create a repository with url "https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-with-sha-512/"


Actual results:
packages are not getting copied in case if the repo is sha-512 

Expected results:
packages should get copied in case if the repo is sha-512 

Additional info:
Comment 3 Justin Sherrill 2020-04-06 14:05:53 UTC 
I can reproduce on the latest 6.7: 

[root@sat-6-7-qa-rhel7 ~]# rpm -qa | grep pulp | sort
libmodulemd-1.7.0-1.pulp.el7sat.x86_64
libsolv-0.7.4-4.pulp.el7sat.x86_64
pulp-client-1.0-1.noarch
pulp-docker-plugins-3.2.5-1.el7sat.noarch
pulp-katello-1.0.3-1.el7sat.noarch
pulp-maintenance-2.21.0-1.el7sat.noarch
pulp-ostree-plugins-1.3.1-2.el7sat.noarch
pulp-puppet-plugins-2.21.0-1.el7sat.noarch
pulp-puppet-tools-2.21.0-1.el7sat.noarch
pulp-rpm-plugins-2.21.0.4-1.el7sat.noarch
pulp-selinux-2.21.0-1.el7sat.noarch
pulp-server-2.21.0-1.el7sat.noarch
python2-solv-0.7.4-4.pulp.el7sat.x86_64
python-pulp-common-2.21.0-1.el7sat.noarch
python-pulp-docker-common-3.2.5-1.el7sat.noarch
python-pulp-integrity-2.21.0.4-1.el7sat.noarch
python-pulp-oid_validation-2.21.0-1.el7sat.noarch
python-pulp-ostree-common-1.3.1-2.el7sat.noarch
python-pulp-puppet-common-2.21.0-1.el7sat.noarch
python-pulp-repoauth-2.21.0-1.el7sat.noarch
python-pulp-rpm-common-2.21.0.4-1.el7sat.noarch
python-pulp-streamer-2.21.0-1.el7sat.noarch



Pulp throws this in the logs:

Download succeeded: https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-with-sha-512/zebra-0.1-2.noarch.rpm.
ERROR: (8377-95616) Listener error on event: succeeded
ERROR: (8377-95616) Traceback (most recent call last):
ERROR: (8377-95616)   File "/usr/lib/python2.7/site-packages/pulp/server/content/sources/event.py", line 39, in __call__
ERROR: (8377-95616)     listener.on_event(self)
ERROR: (8377-95616)   File "/usr/lib/python2.7/site-packages/pulp/server/content/sources/event.py", line 135, in on_event
ERROR: (8377-95616)     method(event.request)
ERROR: (8377-95616)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/repomd/alternate.py", line 119, in on_succeeded
ERROR: (8377-95616)     self.content_listener.download_succeeded(report)
ERROR: (8377-95616)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/listener.py", line 195, in download_succeeded
ERROR: (8377-95616)     super(RPMListener, self).download_succeeded(report)
ERROR: (8377-95616)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/listener.py", line 86, in download_succeeded
ERROR: (8377-95616)     self._verify_checksum(unit, report.destination)
ERROR: (8377-95616)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/listener.py", line 150, in _verify_checksum
ERROR: (8377-95616)     if sums[unit.checksumtype] != unit.checksum:
ERROR: (8377-95616) KeyError: u'sha512'



Strangely, on an older 6.7 snap it was fine, and the only difference in pulp versions was pulp-rpm-plugins-2.21.0.4-1.el7sat.noarch  vs pulp-rpm-plugins-2.21.0.3-1.el7sat.noarch

Moving to the pulp component
Comment 4 Justin Sherrill 2020-04-06 14:07:10 UTC 
Also, to clarify, the full reproducer steps:


Steps to Reproduce:
1. create a repository with url "https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-with-sha-512/"
2. sync the repository
3. check the rpm count in the repo
Comment 6 Justin Sherrill 2020-04-06 16:34:10 UTC 
After some investigation tanya and I discovered:

* This is not a regression, the issue happens on 6.6 if you create the custom repository's download_policy as 'immediate'
* In 6.7, the default download policy for custom repos was changed from on_demand to immediate
* Any upgrade from 6.6 will keep that default of 'on_demand', hence why i didn't see the error on an upgraded satellite
* This means that likely on_demand repositories are NOT having their content validated upon download, but this may need some more investigation 

But in summary:

This is not a regression, and should not break any existing users that are upgrading to satellite 6.7.  Users that do a fresh install of satellite 6.7 can workaround the issue by switching to on demand for their download policy.
Comment 7 Sudhir Mallamprabhakara 2020-04-06 16:46:40 UTC 
Omkar, Please test and confirm.

-Sudhir
Comment 8 Omkar Khatavkar 2020-04-07 06:05:05 UTC 
Not a regression issue as this exists in Satellite 6.6 with if Download Policy is Immediate. Making it them to on-demand workaround works. As Default Custom Repository download policy did not exist earlier version in 6.6 and making default value as 'Immediate' causing issue.
Comment 9 Mike McCune 2020-04-07 15:12:20 UTC 
We are moving this out of 6.7 as it is not a regression. From Justin Sherrill:

"""
After some investigation Tanya and I discovered:

* This is not a regression, the issue happens on 6.6 if you create the custom repository's download_policy as 'immediate'
* In 6.7, the default download policy for custom repos was changed from on_demand to immediate
* Any upgrade from 6.6 will keep that default of 'on_demand', hence why i didn't see the error on an upgraded satellite
* This means that likely on_demand repositories are NOT having their content validated upon download, but this may need some more investigation

But in summary:

This is not a regression, and should not break any existing users that are upgrading to satellite 6.7.  Users that do a fresh install of Satellite 6.7 can workaround the issue by switching to on demand for their download policy.
""
Comment 10 pulp-infra@redhat.com 2020-05-08 19:30:35 UTC 
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.
Comment 11 pulp-infra@redhat.com 2020-05-08 19:30:36 UTC 
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.
Comment 12 pulp-infra@redhat.com 2020-05-08 22:14:08 UTC 
All upstream Pulp bugs are at MODIFIED+. Moving this bug to POST.
Comment 15 pulp-infra@redhat.com 2020-05-19 14:31:48 UTC 
The Pulp upstream bug status is at ON_QA. Updating the external tracker on this bug.
Comment 16 pulp-infra@redhat.com 2020-05-19 16:32:38 UTC 
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.
Comment 22 errata-xmlrpc 2020-10-27 13:01:16 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.8 release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:4366
Note You need to log in before you can comment on or make changes to this bug.