Bug 1347364

Description of problem:
There is an KeyError with unprintable characters generated when syncing CDN channel ed_Hat_Satellite-Red_Hat_Satellite_6_1_for_RHEL_6_Server_RPMs_x86_64.

There is already an upstream bug https://pulp.plan.io/issues/1932 that mentions to handle such a failures more graceful, even the upstream bug has a trace attached for the same channel on '2016-05-23 10:47:37 CEST'

---
Jun 16 12:43:47 li-lc-1578 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Generating metadata databases.
Jun 16 12:43:51 li-lc-1578 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Determining which units need to be downloaded.
Jun 16 12:44:06 li-lc-1578 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (18358-52704) '\xff\xff\xff\xffX\xbb\x8d\x04'
Jun 16 12:44:06 li-lc-1578 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (18358-52704) Traceback (most recent call last):
Jun 16 12:44:06 li-lc-1578 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (18358-52704)   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 220, in run
Jun 16 12:44:06 li-lc-1578 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (18358-52704)     self.update_content(metadata_files, url)
Jun 16 12:44:06 li-lc-1578 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (18358-52704)   File "/usr/lib64/python2.6/contextlib.py", line 34, in __exit__
Jun 16 12:44:06 li-lc-1578 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (18358-52704)     self.gen.throw(type, value, traceback)
Jun 16 12:44:06 li-lc-1578 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (18358-52704)   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 183, in update_state
Jun 16 12:44:06 li-lc-1578 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (18358-52704)     yield skip
Jun 16 12:44:06 li-lc-1578 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (18358-52704)   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 220, in run
Jun 16 12:44:06 li-lc-1578 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (18358-52704)     self.update_content(metadata_files, url)
Jun 16 12:44:06 li-lc-1578 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (18358-52704)   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 468, in update_content
Jun 16 12:44:06 li-lc-1578 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (18358-52704)     self.download_rpms(metadata_files, rpms_to_download, url)
Jun 16 12:44:06 li-lc-1578 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (18358-52704)   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 660, in download_rpms
Jun 16 12:44:06 li-lc-1578 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (18358-52704)     self.add_rpm_unit(metadata_files, unit)
Jun 16 12:44:06 li-lc-1578 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (18358-52704)   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 612, in add_rpm_unit
Jun 16 12:44:06 li-lc-1578 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (18358-52704)     metadata_files.add_repodata(unit)
Jun 16 12:44:06 li-lc-1578 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (18358-52704)   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/repomd/metadata.py", line 338, in add_repodata
Jun 16 12:44:06 li-lc-1578 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (18358-52704)     raw_xml = db_file[db_key]
Jun 16 12:44:06 li-lc-1578 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (18358-52704) KeyError: '\xff\xff\xff\xffX\xbb\x8d\x04'
Jun 16 12:44:06 li-lc-1578 pulp: pulp.server.event.http:INFO: (18358-52704) {"call_report": {"exception": null, "task_type": "pulp.server.managers.repo.sync.sync", "_href": "/pulp/api/v2/tasks/6ce33fc7-40c9-48d8-ace8-53b7fbe5daa4/", "task_id": "6ce33fc7-40c9-48d8-ace8-53b7fbe5daa4", "tags": ["pulp:repository:Hilti-Red_Hat_Satellite-Red_Hat_Satellite_6_1_for_RHEL_6_Server_RPMs_x86_64", "pulp:action:sync"], "finish_time": null, "_ns": "task_status", "start_time": "2016-06-16T12:43:42Z", "traceback": null, "spawned_tasks": [], "progress_report": {"yum_importer": {"content": {"size_total": 730103283, "items_left": 46, "items_total": 569, "state": "FAILED", "size_left": 35044717, "details": {"rpm_total": 569, "rpm_done": 523, "drpm_total": 0, "drpm_done": 0}, "error": "'\\xff\\xff\\xff\\xffX\\xbb\\x8d\\x04'", "error_details": []}, "comps": {"state": "NOT_STARTED"}, "purge_duplicates": {"state": "NOT_STARTED"}, "distribution": {"items_total": 0, "state": "NOT_STARTED", "error_details": [], "items_left": 0}, "errata": {"state": "NOT_STARTED"}, "metadata": {"state": "FINISHED"}}}, "state": "running", "worker_name": "reserved_resource_worker-5@li-lc-1578", "result": null, "error": null, "_id": {"$oid": "57629ab888d766793efb3348"}, "id": "57629ab888d766793efb3348"}, "event_type": "repo.sync.finish", "payload": {"importer_id": "yum_importer", "exception": null, "repo_id": "Hilti-Red_Hat_Satellite-Red_Hat_Satellite_6_1_for_RHEL_6_Server_RPMs_x86_64", "traceback": null, "started": "2016-06-16T12:43:42Z", "_ns": "repo_sync_results", "completed": "2016-06-16T12:44:06Z", "importer_type_id": "yum_importer", "error_message": null, "summary": {"content": {"state": "FAILED"}, "comps": {"state": "NOT_STARTED"}, "purge_duplicates": {"state": "NOT_STARTED"}, "distribution": {"state": "NOT_STARTED"}, "errata": {"state": "NOT_STARTED"}, "metadata": {"state": "FINISHED"}}, "added_count": 615, "result": "failed", "updated_count": 0, "details": {"content": {"size_total": 730103283, "items_left": 46, "items_total": 569, "state": "FAILED", "size_left": 35044717,
Jun 16 12:44:06 li-lc-1578 pulp: pulp.server.event.http:INFO: (18358-52704)  "details": {"rpm_total": 569, "rpm_done": 523, "drpm_total": 0, "drpm_done": 0}, "error": "'\\xff\\xff\\xff\\xffX\\xbb\\x8d\\x04'", "error_details": []}, "comps": {"state": "NOT_STARTED"}, "purge_duplicates": {"state": "NOT_STARTED"}, "distribution": {"items_total": 0, "state": "NOT_STARTED", "error_details": [], "items_left": 0}, "errata": {"state": "NOT_STARTED"}, "metadata": {"state": "FINISHED"}}, "id": "57629f160586fe47b6e75984", "removed_count": 0}}
Jun 16 12:44:06 li-lc-1578 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): li-lc-1578.hag.hilti.com
Jun 16 12:44:06 li-lc-1578 pulp: pulp.server.async.tasks:INFO: Task failed : [6ce33fc7-40c9-48d8-ace8-53b7fbe5daa4]
Jun 16 12:44:06 li-lc-1578 pulp: celery.worker.job:ERROR: (18257-52704) Task pulp.server.managers.repo.sync.sync[6ce33fc7-40c9-48d8-ace8-53b7fbe5daa4] raised unexpected: PulpExecutionException('Importer indicated a failed response',)
Jun 16 12:44:06 li-lc-1578 pulp: celery.worker.job:ERROR: (18257-52704) Traceback (most recent call last):
Jun 16 12:44:06 li-lc-1578 pulp: celery.worker.job:ERROR: (18257-52704)   File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 240, in trace_task
Jun 16 12:44:06 li-lc-1578 pulp: celery.worker.job:ERROR: (18257-52704)     R = retval = fun(*args, **kwargs)
Jun 16 12:44:06 li-lc-1578 pulp: celery.worker.job:ERROR: (18257-52704)   File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 473, in __call__
Jun 16 12:44:06 li-lc-1578 pulp: celery.worker.job:ERROR: (18257-52704)     return super(Task, self).__call__(*args, **kwargs)
Jun 16 12:44:06 li-lc-1578 pulp: celery.worker.job:ERROR: (18257-52704)   File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 103, in __call__
Jun 16 12:44:06 li-lc-1578 pulp: celery.worker.job:ERROR: (18257-52704)     return super(PulpTask, self).__call__(*args, **kwargs)
Jun 16 12:44:06 li-lc-1578 pulp: celery.worker.job:ERROR: (18257-52704)   File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in __protected_call__
Jun 16 12:44:06 li-lc-1578 pulp: celery.worker.job:ERROR: (18257-52704)     return self.run(*args, **kwargs)
Jun 16 12:44:06 li-lc-1578 pulp: celery.worker.job:ERROR: (18257-52704)   File "/usr/lib/python2.6/site-packages/pulp/server/controllers/repository.py", line 810, in sync
Jun 16 12:44:06 li-lc-1578 pulp: celery.worker.job:ERROR: (18257-52704)     raise pulp_exceptions.PulpExecutionException(_('Importer indicated a failed response'))
Jun 16 12:44:06 li-lc-1578 pulp: celery.worker.job:ERROR: (18257-52704) PulpExecutionException: Importer indicated a failed response
---



Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info: