Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1337662

Summary: repo sync caused an KeyError: '\xff\xff\xff\xff\xd8\xec2\x05' in pulp
Product: Red Hat Satellite Reporter: dgao
Component: PulpAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED DUPLICATE QA Contact: Katello QA List <katello-qa-list>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.2.0CC: bbuckingham, bmbouter, daviddavis, dkliban, ggainey, ipanova, jmontleo, mhrivnak, pcreech, peter.vreman, rchan, tsanders, ttereshc
Target Milestone: UnspecifiedKeywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-12-12 13:47:28 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1122832, 1212602    

Description dgao 2016-05-19 18:37:36 UTC
Encountered the following error while trying to sync Red_Hat_Enterprise_Virtualization_Management_Agents_for_RHEL_7_RPMs_x86_64_7Server 

May 19 10:30:02 localhost pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Determining which units need to be downloaded.
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512) '\xff\xff\xff\xff\xd8\xec2\x05'
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512) Traceback (most recent call last):
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 220, in run
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512)     self.update_content(metadata_files, url)
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 451, in update_content
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512)     self.download_rpms(metadata_files, rpms_to_download, url)
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 643, in download_rpms
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512)     self.add_rpm_unit(metadata_files, unit)
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 595, in add_rpm_unit
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512)     metadata_files.add_repodata(unit)
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/repomd/metadata.py", line 338, in add_repodata
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512)     raw_xml = db_file[db_key]
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512) KeyError: '\xff\xff\xff\xff\xd8\xec2\x05'


Full Error: http://paste-platops.itos.redhat.com/prqdzun2x/2yncaj

Comment 1 dgao 2016-05-19 18:38:03 UTC
This issue is being tracked in pulp under: https://pulp.plan.io/issues/1932

Comment 2 Jason Montleon 2016-05-19 19:17:25 UTC
It seems possible to reproduce this with a clean Satellite, upgrading pulp to 2.8.3, and setting the download policy to on_demand.

Changing it to immediate will allow the repo to sync, and then even on_demand will complete without error.

Comment 4 Brad Buckingham 2016-06-17 15:33:05 UTC
*** Bug 1347364 has been marked as a duplicate of this bug. ***

Comment 5 Michael Hrivnak 2016-06-20 16:57:54 UTC
I determined that although pulp's error reporting should improve in two different ways, the error it is reporting poorly is caused by missing data on the CDN. There is not a behavioral problem in pulp causing the error.

See the upstream pulp bug for details, particularly comment 11:

https://pulp.plan.io/issues/1932#note-11

Comment 6 pulp-infra@redhat.com 2016-11-21 18:54:16 UTC
The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.

Comment 7 pulp-infra@redhat.com 2016-11-21 18:54:19 UTC
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 8 Michael Hrivnak 2016-12-12 13:47:28 UTC

*** This bug has been marked as a duplicate of bug 1267579 ***

Comment 9 pulp-infra@redhat.com 2016-12-12 14:02:13 UTC
The Pulp upstream bug status is at CLOSED - DUPLICATE. Updating the external tracker on this bug.