Description of problem: A client reported failures while attempting to import the EPEL5 repo into their satellite. I had a look and replicated the issue using pulp directly to simplify testing. The issue appears to be that the updateinfo errata metadata in the repo doesn't have "description" elements defined for all of the erratum. The pulp-rpm importer doesn't much like that and fails. Version-Release number of selected component (if applicable): How reproducible: Always Steps to Reproduce: 1. pulp-admin rpm repo create --repo-id=test_epel5_pulp --display-name="RHEL5 EPEL" --feed=https://dl.fedoraproject.org/pub/epel/5/x86_64 --relative-url=epel5 --serve-http=true --serve-https=true 2. pulp-admin rpm repo sync run --repo-id=test_epel5_pulp 3. Fail. Actual results: [root@satellite updateinfotest]# pulp-admin rpm repo create --repo-id=test_epel5_pulp --display-name="RHEL5 EPEL" --feed=https://dl.fedoraproject.org/pub/epel/5/x86_64 --relative-url=epel5 --serve-http=true --serve-https=true Successfully created repository [test_epel5_pulp] [root@satellite updateinfotest]# pulp-admin rpm repo sync run --repo-id=test_epel5_pulp +----------------------------------------------------------------------+ Synchronizing Repository [test_epel5_pulp] +----------------------------------------------------------------------+ This command may be exited via ctrl+c without affecting the request. Downloading metadata... [-] ... completed Downloading repository content... [==================================================] 100% RPMs: 0/0 items Delta RPMs: 0/0 items ... completed Downloading distribution files... [==================================================] 100% Distributions: 0/0 items ... completed Importing errata... [-] ... failed Task Failed Importer indicated a failed response <log> Jan 29 22:51:15 satellite pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_reserved_task[e5bdb13c-4135-4eee-9583-87a783940328] Jan 29 22:51:15 satellite pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.sync.sync[0aa038fc-4885-49e2-965a-894a1a067f8b] Jan 29 22:51:15 satellite pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading metadata from https://dl.fedoraproject.org/pub/epel/5/x86_64/. Jan 29 22:51:15 satellite pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): dl.fedoraproject.org Jan 29 22:51:15 satellite pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[e5bdb13c-4135-4eee-9583-87a783940328] succeeded in 0.0381081299856s: None Jan 29 22:51:16 satellite pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[af88ae8a-7aeb-47af-8d60-c50c22246ac6] Jan 29 22:51:16 satellite pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Parsing metadata. Jan 29 22:51:16 satellite pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading metadata files. Jan 29 22:51:17 satellite pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): dl.fedoraproject.org Jan 29 22:51:17 satellite pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): dl.fedoraproject.org Jan 29 22:51:17 satellite pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): dl.fedoraproject.org Jan 29 22:51:17 satellite pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): dl.fedoraproject.org Jan 29 22:51:17 satellite pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): dl.fedoraproject.org Jan 29 22:51:20 satellite pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Generating metadata databases. Jan 29 22:52:00 satellite pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Determining which units need to be downloaded. Jan 29 22:54:52 satellite pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading 0 RPMs. Jan 29 22:55:11 satellite pulp: pulp_rpm.plugins.importers.yum.repomd.alternate:INFO: The content container reported: {'downloads': {}, 'total_sources': 0} for base URL: https://dl.fedoraproject.org/pub/epel/5/x86_64/ Jan 29 22:55:11 satellite pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading additional units. Jan 29 22:55:11 satellite pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): dl.fedoraproject.org Jan 29 22:55:12 satellite pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): dl.fedoraproject.org Jan 29 22:55:14 satellite pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (2977-19136) sync failed Jan 29 22:55:14 satellite pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (2977-19136) Traceback (most recent call last): Jan 29 22:55:14 satellite pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (2977-19136) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 159, in run Jan 29 22:55:14 satellite pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (2977-19136) self.get_errata(metadata_files) Jan 29 22:55:14 satellite pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (2977-19136) File "/usr/lib64/python2.6/contextlib.py", line 34, in __exit__ Jan 29 22:55:14 satellite pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (2977-19136) self.gen.throw(type, value, traceback) Jan 29 22:55:14 satellite pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (2977-19136) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 118, in update_state Jan 29 22:55:14 satellite pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (2977-19136) yield skip Jan 29 22:55:14 satellite pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (2977-19136) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 159, in run Jan 29 22:55:14 satellite pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (2977-19136) self.get_errata(metadata_files) Jan 29 22:55:14 satellite pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (2977-19136) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 522, in get_errata Jan 29 22:55:14 satellite pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (2977-19136) updateinfo.process_package_element, additive_type=True) Jan 29 22:55:14 satellite pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (2977-19136) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 602, in save_fileless_units Jan 29 22:55:14 satellite pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (2977-19136) for model in package_info_generator: Jan 29 22:55:14 satellite pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (2977-19136) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/repomd/packages.py", line 63, in package_list_generator Jan 29 22:55:14 satellite pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (2977-19136) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/repomd/updateinfo.py", line 27, in process_package_element Jan 29 22:55:14 satellite pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (2977-19136) AttributeError: 'NoneType' object has no attribute 'text' Jan 29 22:55:14 satellite pulp: pulp.server.event.http:INFO: (2977-19136) {'call_report': {u'exception': None, u'task_type': u'pulp.server.managers.repo.sync.sync', u'task_id': u'0aa038fc-4885-49e2-965a-894a1a067f8b', u'tags': [u'pulp:repository:test_epel5_pulp', u'pulp:action:sync'], u'finish_time': None, u'_ns': u'task_status', u'start_time': u'2016-01-30T03:51:15Z', u'traceback': None, u'spawned_tasks': [], u'progress_report': {u'yum_importer': {u'content': {u'size_total': 0, u'items_left': 0, u'items_total': 0, u'state': u'FINISHED', u'size_left': 0, u'details': {u'rpm_total': 0, u'rpm_done': 0, u'drpm_total': 0, u'drpm_done': 0}, u'error_details': []}, u'comps': {u'state': u'NOT_STARTED'}, u'distribution': {u'items_total': 0, u'state': u'FINISHED', u'error_details': [], u'items_left': 0}, u'errata': {u'state': u'FAILED', u'error': u"'NoneType' object has no attribute 'text'"}, u'metadata': {u'state': u'FINISHED'}}}, u'state': u'running', u'worker_name': u'reserved_resource_worker-0.us', u'result': None, u'error': None, u'_id': ObjectId('56ac3333295511b21ab954d7'), u'id': u'56ac3333c024450b6e4db11b'}, 'event_type': 'repo.sync.finish', 'payload': {'importer_id': u'yum_importer', 'exception': None, 'repo_id': u'test_epel5_pulp', 'traceback': None, 'started': '2016-01-30T03:51:15Z', '_ns': u'repo_sync_results', 'completed': '2016-01-30T03:55:14Z', 'importer_type_id': u'yum_importer', 'error_message': None, 'summary': {'content': {'state': 'FINISHED'}, 'comps': {'state': 'NOT_STARTED'}, 'distribution': {'state': 'FINISHED'}, 'errata': {'state': 'FAILED'}, 'metadata': {'state': 'FINISHED'}}, 'added_count': 0, 'result': 'failed', 'updated_count': 6781, 'details': {'content': {'size_total': 0, 'items_left': 0, 'items_total': 0, 'state': 'FINISHED', 'size_left': 0, 'details': {'rpm_total': 0, 'rpm_done': 0, 'drpm_total': 0, 'drpm_done': 0}, 'error_details': []}, 'comps': {'state': 'NOT_STARTED'}, 'distribution': {'items_total': 0, 'state': 'FINISHED', 'error_details': [], 'items_left': 0}, 'errata': {'state': 'FAILED', Jan 29 22:55:14 satellite pulp: pulp.server.event.http:INFO: (2977-19136) 'error': "'NoneType' object has no attribute 'text'"}, 'metadata': {'state': 'FINISHED'}}, 'id': '56ac3422c024450ba1aa5bc6', 'removed_count': 0}} Jan 29 22:55:14 satellite pulp: celery.worker.job:ERROR: (2941-19136) Task pulp.server.managers.repo.sync.sync[0aa038fc-4885-49e2-965a-894a1a067f8b] raised unexpected: PulpExecutionException('Importer indicated a failed response',) Jan 29 22:55:14 satellite pulp: celery.worker.job:ERROR: (2941-19136) Traceback (most recent call last): Jan 29 22:55:14 satellite pulp: celery.worker.job:ERROR: (2941-19136) File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 240, in trace_task Jan 29 22:55:14 satellite pulp: celery.worker.job:ERROR: (2941-19136) R = retval = fun(*args, **kwargs) Jan 29 22:55:14 satellite pulp: celery.worker.job:ERROR: (2941-19136) File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 328, in __call__ Jan 29 22:55:14 satellite pulp: celery.worker.job:ERROR: (2941-19136) return super(Task, self).__call__(*args, **kwargs) Jan 29 22:55:14 satellite pulp: celery.worker.job:ERROR: (2941-19136) File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in __protected_call__ Jan 29 22:55:14 satellite pulp: celery.worker.job:ERROR: (2941-19136) return self.run(*args, **kwargs) Jan 29 22:55:14 satellite pulp: celery.worker.job:ERROR: (2941-19136) File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/sync.py", line 114, in sync Jan 29 22:55:14 satellite pulp: celery.worker.job:ERROR: (2941-19136) raise PulpExecutionException(_('Importer indicated a failed response')) Jan 29 22:55:14 satellite pulp: celery.worker.job:ERROR: (2941-19136) PulpExecutionException: Importer indicated a failed response Jan 29 22:55:14 satellite pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[af88ae8a-7aeb-47af-8d60-c50c22246ac6] succeeded in 0.011311418988s: None Jan 29 22:55:14 satellite pulp: pulp.server.event.http:WARNING: Error response from HTTP notifier: {"displayMessage":"Couldn't find repository 'test_epel5_pulp'","errors":["Couldn't find repository 'test_epel5_pulp'"]} </log> Expected results: Works. Additional info: Having a look at the updateinfo for EPEL5: <epel5> [fmhirtz@x1 epel5]$ wget -q https://dl.fedoraproject.org/pub/epel/5/x86_64/repodata/c0f4c9f3e20d7aa48e98b37e9cdf713ed9060280-updateinfo.xml.gz[fmhirtz@x1 epel5]$ ls c0f4c9f3e20d7aa48e98b37e9cdf713ed9060280-updateinfo.xml.gz [fmhirtz@x1 epel5]$ zgrep '<id>' c0f4c9f3e20d7aa48e98b37e9cdf713ed9060280-updateinfo.xml.gz | wc -l 1877 [fmhirtz@x1 epel5]$ zgrep '<name>' c0f4c9f3e20d7aa48e98b37e9cdf713ed9060280-updateinfo.xml.gz | wc -l 1877 [fmhirtz@x1 epel5]$ zgrep '<description>' c0f4c9f3e20d7aa48e98b37e9cdf713ed9060280-updateinfo.xml.gz | wc -l 1652 [fmhirtz@x1 epel5]$ </epel5> As a sanity check, EPEL6 doesn't have any such discrepancy: <epel6 (works)> [fmhirtz@x1 epel6]$ wget -q https://dl.fedoraproject.org/pub/epel/6/x86_64/repodata/7b7192d8e50b4613e2cca15a0a4875d6e985a8a96077151f7eebea7681831596-updateinfo.xml.bz2 [fmhirtz@x1 epel6]$ ls 7b7192d8e50b4613e2cca15a0a4875d6e985a8a96077151f7eebea7681831596-updateinfo.xml.bz2 [fmhirtz@x1 epel6]$ bzcat 7b7192d8e50b4613e2cca15a0a4875d6e985a8a96077151f7eebea7681831596-updateinfo.xml.bz2 | grep '<id>' | wc -l 3826 [fmhirtz@x1 epel6]$ bzcat 7b7192d8e50b4613e2cca15a0a4875d6e985a8a96077151f7eebea7681831596-updateinfo.xml.bz2 | grep '<name>' | wc -l 3826 [fmhirtz@x1 epel6]$ bzcat 7b7192d8e50b4613e2cca15a0a4875d6e985a8a96077151f7eebea7681831596-updateinfo.xml.bz2 | grep '<description>' | wc -l 3826 [fmhirtz@x1 epel6]$ </epel6>
This was fixed upstream in pulp 2.6.1 and can be cherry-picked on a sat 6.1 Z-stream if desired. I added the link to the upstream issue.
based on comment 2, I am going to move this to ON_QA as part of 6.2
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2016:1501