Bug 1337662

Summary:	repo sync caused an KeyError: '\xff\xff\xff\xff\xd8\xec2\x05' in pulp
Product:	Red Hat Satellite	Reporter:	dgao
Component:	Pulp	Assignee:	satellite6-bugs <satellite6-bugs>
Status:	CLOSED DUPLICATE	QA Contact:	Katello QA List <katello-qa-list>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	6.2.0	CC:	bbuckingham, bmbouter, daviddavis, dkliban, ggainey, ipanova, jmontleo, mhrivnak, pcreech, peter.vreman, rchan, tsanders, ttereshc
Target Milestone:	Unspecified	Keywords:	Triaged
Target Release:	Unused
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-12-12 13:47:28 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1122832, 1212602

Description dgao 2016-05-19 18:37:36 UTC

Encountered the following error while trying to sync Red_Hat_Enterprise_Virtualization_Management_Agents_for_RHEL_7_RPMs_x86_64_7Server 

May 19 10:30:02 localhost pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Determining which units need to be downloaded.
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512) '\xff\xff\xff\xff\xd8\xec2\x05'
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512) Traceback (most recent call last):
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 220, in run
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512)     self.update_content(metadata_files, url)
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 451, in update_content
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512)     self.download_rpms(metadata_files, rpms_to_download, url)
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 643, in download_rpms
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512)     self.add_rpm_unit(metadata_files, unit)
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 595, in add_rpm_unit
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512)     metadata_files.add_repodata(unit)
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/repomd/metadata.py", line 338, in add_repodata
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512)     raw_xml = db_file[db_key]
May 19 10:30:08 localhost pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (3528-08512) KeyError: '\xff\xff\xff\xff\xd8\xec2\x05'


Full Error: http://paste-platops.itos.redhat.com/prqdzun2x/2yncaj

Comment 1 dgao 2016-05-19 18:38:03 UTC

This issue is being tracked in pulp under: https://pulp.plan.io/issues/1932

Comment 2 Jason Montleon 2016-05-19 19:17:25 UTC

It seems possible to reproduce this with a clean Satellite, upgrading pulp to 2.8.3, and setting the download policy to on_demand.

Changing it to immediate will allow the repo to sync, and then even on_demand will complete without error.

Comment 4 Brad Buckingham 2016-06-17 15:33:05 UTC

*** Bug 1347364 has been marked as a duplicate of this bug. ***

Comment 5 Michael Hrivnak 2016-06-20 16:57:54 UTC

I determined that although pulp's error reporting should improve in two different ways, the error it is reporting poorly is caused by missing data on the CDN. There is not a behavioral problem in pulp causing the error.

See the upstream pulp bug for details, particularly comment 11:

https://pulp.plan.io/issues/1932#note-11

Comment 6 pulp-infra@redhat.com 2016-11-21 18:54:16 UTC

The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.

Comment 7 pulp-infra@redhat.com 2016-11-21 18:54:19 UTC

The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 8 Michael Hrivnak 2016-12-12 13:47:28 UTC


*** This bug has been marked as a duplicate of bug 1267579 ***

Comment 9 pulp-infra@redhat.com 2016-12-12 14:02:13 UTC

The Pulp upstream bug status is at CLOSED - DUPLICATE. Updating the external tracker on this bug.