Description of problem: Trying to mirror Fedora EPEL I receive the following error: Importing errata... [\] ... failed Version-Release number of selected component (if applicable): pulp 2.4.3 All installed RPMs (on the server): pulp-rpm-plugins-2.4.3-1.el6.noarch pulp-selinux-2.4.3-1.el6.noarch python-pulp-bindings-2.4.3-1.el6.noarch pulp-admin-client-2.4.3-1.el6.noarch pulp-puppet-admin-extensions-2.4.3-1.el6.noarch python-isodate-0.5.0-4.pulp.el6.noarch python-pulp-rpm-common-2.4.3-1.el6.noarch python-kombu-3.0.15-12.pulp.el6.noarch pulp-server-2.4.3-1.el6.noarch pulp-puppet-plugins-2.4.3-1.el6.noarch python-pulp-client-lib-2.4.3-1.el6.noarch pulp-rpm-admin-extensions-2.4.3-1.el6.noarch python-pulp-common-2.4.3-1.el6.noarch python-rhsm-1.8.0-2.pulp.el6.x86_64 python-pulp-puppet-common-2.4.3-1.el6.noarch mod_wsgi-3.4-1.pulp.el6.x86_64 How reproducible: Create the repo: pulp-admin rpm repo create --repo-id=foo_epel --relative-url=foo --feed=http://ftp.linux.ncsu.edu/pub/epel/6/x86_64 Sync: pulp-admin --debug rpm repo sync run --repo-id=foo_epel Actual results: +----------------------------------------------------------------------+ Synchronizing Repository [foo_epel] +----------------------------------------------------------------------+ This command may be exited via ctrl+c without affecting the request. Downloading metadata... [|] ... completed Downloading repository content... [==================================================] 100% RPMs: 0/0 items Delta RPMs: 0/0 items ... completed Downloading distribution files... [==================================================] 100% Distributions: 0/0 items ... completed Importing errata... [\] ... failed Task Failed Importer indicated a failed response Expected results: Successful sync Additional info: Logs: Nov 6 10:56:03 aabd960fe5fb pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._reserve_resource[9a731429-2c86-4e0b-98b0-4d8e66da28d7] Nov 6 10:56:03 aabd960fe5fb pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._reserve_resource[9a731429-2c86-4e0b-98b0-4d8e66da28d7] succeeded in 0.0562103479169s: u'reserved_resource_worker-1' Nov 6 10:56:04 aabd960fe5fb pulp: celery.worker.strategy:INFO: Received task: pulp.server.tasks.repository.sync_with_auto_publish[068d1b1d-0a4f-4d7c-9765-1e9b570710cc] Nov 6 10:56:04 aabd960fe5fb pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_release_resource[ce3975df-de3c-4818-a4c0-c073c18b7026] Nov 6 10:56:04 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading metadata from http://ftp.osuosl.org/pub/fedora-epel/6/x86_64/. Nov 6 10:56:04 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): ftp.osuosl.org Nov 6 10:56:05 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): ftp.osuosl.org Nov 6 10:56:05 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): ftp.osuosl.org Nov 6 10:56:05 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): ftp.osuosl.org Nov 6 10:56:05 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): ftp.osuosl.org Nov 6 10:56:05 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): ftp.osuosl.org Nov 6 10:56:15 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Generating metadata databases. Nov 6 10:57:39 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Determining which units need to be downloaded. Nov 6 10:59:21 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading 30 RPMs. Nov 6 10:59:22 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): ftp.osuosl.org Nov 6 10:59:26 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): ftp.osuosl.org Nov 6 10:59:27 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): ftp.osuosl.org Nov 6 10:59:29 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): ftp.osuosl.org Nov 6 10:59:30 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): ftp.osuosl.org Nov 6 10:59:35 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 10:59:38 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 10:59:39 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 10:59:39 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 10:59:41 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 10:59:41 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 10:59:42 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 10:59:43 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 10:59:43 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 10:59:44 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 10:59:45 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 10:59:46 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 10:59:47 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 10:59:49 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 10:59:50 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 10:59:50 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 10:59:51 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 10:59:53 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 10:59:54 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 10:59:55 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 10:59:56 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 10:59:58 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 11:00:03 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 11:00:06 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 11:00:08 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: ftp.osuosl.org Nov 6 11:00:08 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading additional units. Nov 6 11:00:08 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): ftp.osuosl.org Nov 6 11:00:09 aabd960fe5fb pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): ftp.osuosl.org Nov 6 11:00:14 aabd960fe5fb pulp: pulp.plugins.conduits.mixins:ERROR: Content unit association failed [Unit [key={'id': 'FEDORA-EPEL-2012-5980'}] [type=erratum] [id=None]] Nov 6 11:00:14 aabd960fe5fb pulp: pulp.plugins.conduits.mixins:ERROR: Traceback (most recent call last): Nov 6 11:00:14 aabd960fe5fb pulp: pulp.plugins.conduits.mixins:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/conduits/mixins.py", line 480, in save_unit Nov 6 11:00:14 aabd960fe5fb pulp: pulp.plugins.conduits.mixins:ERROR: unit.id = self._update_unit(unit, pulp_unit) Nov 6 11:00:14 aabd960fe5fb pulp: pulp.plugins.conduits.mixins:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/conduits/mixins.py", line 512, in _update_unit Nov 6 11:00:14 aabd960fe5fb pulp: pulp.plugins.conduits.mixins:ERROR: return self._add_unit(unit, pulp_unit) Nov 6 11:00:14 aabd960fe5fb pulp: pulp.plugins.conduits.mixins:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/conduits/mixins.py", line 534, in _add_unit Nov 6 11:00:14 aabd960fe5fb pulp: pulp.plugins.conduits.mixins:ERROR: unit_id = content_manager.add_content_unit(unit.type_id, None, pulp_unit) Nov 6 11:00:14 aabd960fe5fb pulp: pulp.plugins.conduits.mixins:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/managers/content/cud.py", line 35, in add_content_unit Nov 6 11:00:14 aabd960fe5fb pulp: pulp.plugins.conduits.mixins:ERROR: collection.insert(unit_doc, safe=True) Nov 6 11:00:14 aabd960fe5fb pulp: pulp.plugins.conduits.mixins:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", line 133, in retry Nov 6 11:00:14 aabd960fe5fb pulp: pulp.plugins.conduits.mixins:ERROR: return method(*args, **kwargs) Nov 6 11:00:14 aabd960fe5fb pulp: pulp.plugins.conduits.mixins:ERROR: File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 357, in insert Nov 6 11:00:14 aabd960fe5fb pulp: pulp.plugins.conduits.mixins:ERROR: continue_on_error, self.__uuid_subtype), safe) Nov 6 11:00:14 aabd960fe5fb pulp: pulp.plugins.conduits.mixins:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", line 156, in _with_end_request Nov 6 11:00:14 aabd960fe5fb pulp: pulp.plugins.conduits.mixins:ERROR: return method(*args, **kwargs) Nov 6 11:00:14 aabd960fe5fb pulp: pulp.plugins.conduits.mixins:ERROR: File "/usr/lib64/python2.6/site-packages/pymongo/mongo_client.py", line 920, in _send_message Nov 6 11:00:14 aabd960fe5fb pulp: pulp.plugins.conduits.mixins:ERROR: rv = self.__check_response_to_last_error(response) Nov 6 11:00:14 aabd960fe5fb pulp: pulp.plugins.conduits.mixins:ERROR: File "/usr/lib64/python2.6/site-packages/pymongo/mongo_client.py", line 863, in __check_response_to_last_error Nov 6 11:00:14 aabd960fe5fb pulp: pulp.plugins.conduits.mixins:ERROR: raise OperationFailure(details["err"], details["code"]) Nov 6 11:00:14 aabd960fe5fb pulp: pulp.plugins.conduits.mixins:ERROR: OperationFailure: insertDocument :: caused by :: 17280 Btree::insert: key too large to index, failing pulp_database.units_erratum.$title_1 1267 { : "ghc-7.0.4-46.el6,haskell-platform-2011.4.0.0-6.el6,ghc-rpm-macros-0.15.5-1.el6,hscolour-1.19-6.el6,ghc-HUnit-1.2.4.2-3.el6,ghc-transformers-0.2.2.0-14..." } Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: sync failed Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: Traceback (most recent call last): Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 133, in run Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: self.get_errata(metadata_files) Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 462, in get_errata Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: self.save_fileless_units(errata_file_handle, updateinfo.PACKAGE_TAG, updateinfo.process_package_element) Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 533, in save_fileless_units Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: self.sync_conduit.save_unit(unit) Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/conduits/mixins.py", line 480, in save_unit Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: unit.id = self._update_unit(unit, pulp_unit) Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/conduits/mixins.py", line 512, in _update_unit Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: return self._add_unit(unit, pulp_unit) Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/conduits/mixins.py", line 534, in _add_unit Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: unit_id = content_manager.add_content_unit(unit.type_id, None, pulp_unit) Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/managers/content/cud.py", line 35, in add_content_unit Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: collection.insert(unit_doc, safe=True) Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", line 133, in retry Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: return method(*args, **kwargs) Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 357, in insert Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: continue_on_error, self.__uuid_subtype), safe) Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", line 156, in _with_end_request Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: return method(*args, **kwargs) Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: File "/usr/lib64/python2.6/site-packages/pymongo/mongo_client.py", line 920, in _send_message Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: rv = self.__check_response_to_last_error(response) Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: File "/usr/lib64/python2.6/site-packages/pymongo/mongo_client.py", line 863, in __check_response_to_last_error Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: raise OperationFailure(details["err"], details["code"]) Nov 6 11:00:14 aabd960fe5fb pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: ImporterConduitException: insertDocument :: caused by :: 17280 Btree::insert: key too large to index, failing pulp_database.units_erratum.$title_1 1267 { : "ghc-7.0.4-46.el6,haskell-platform-2011.4.0.0-6.el6,ghc-rpm-macros-0.15.5-1.el6,hscolour-1.19-6.el6,ghc-HUnit-1.2.4.2-3.el6,ghc-transformers-0.2.2.0-14..." } Nov 6 11:00:15 aabd960fe5fb pulp: celery.worker.job:ERROR: Task pulp.server.tasks.repository.sync_with_auto_publish[068d1b1d-0a4f-4d7c-9765-1e9b570710cc] raised unexpected: PulpExecutionException('Importer indicated a failed response',) Nov 6 11:00:15 aabd960fe5fb pulp: celery.worker.job:ERROR: Traceback (most recent call last): Nov 6 11:00:15 aabd960fe5fb pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 240, in trace_task Nov 6 11:00:15 aabd960fe5fb pulp: celery.worker.job:ERROR: R = retval = fun(*args, **kwargs) Nov 6 11:00:15 aabd960fe5fb pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 306, in __call__ Nov 6 11:00:15 aabd960fe5fb pulp: celery.worker.job:ERROR: return super(Task, self).__call__(*args, **kwargs) Nov 6 11:00:15 aabd960fe5fb pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in __protected_call__ Nov 6 11:00:15 aabd960fe5fb pulp: celery.worker.job:ERROR: return self.run(*args, **kwargs) Nov 6 11:00:15 aabd960fe5fb pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/tasks/repository.py", line 210, in sync_with_auto_publish Nov 6 11:00:15 aabd960fe5fb pulp: celery.worker.job:ERROR: sync_result = managers.repo_sync_manager().sync(repo_id, sync_config_override=overrides) Nov 6 11:00:15 aabd960fe5fb pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/sync.py", line 113, in sync Nov 6 11:00:15 aabd960fe5fb pulp: celery.worker.job:ERROR: raise PulpExecutionException(_('Importer indicated a failed response')) Nov 6 11:00:15 aabd960fe5fb pulp: celery.worker.job:ERROR: PulpExecutionException: Importer indicated a failed response Nov 6 11:00:15 aabd960fe5fb pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[74118024-99a0-4a75-a2e3-b40b09bda87e] Nov 6 11:00:15 aabd960fe5fb pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_release_resource[ce3975df-de3c-4818-a4c0-c073c18b7026] succeeded in 0.0267922324128s: None Nov 6 11:00:15 aabd960fe5fb pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[74118024-99a0-4a75-a2e3-b40b09bda87e] succeeded in 0.0531482887454s: None Additional notes: I've tried several different mirrors on the assumption the mirror was missing errata, all mirrors had the same issue.
This appears to have been a bug in the errata that was published that has since been fixed. I was able to download and publish the errata from http://ftp.linux.ncsu.edu/pub/epel/6/x86_64 as well as http://dl.fedoraproject.org/pub/epel/6/x86_64/
I've just hit the same with http://ftp.fi.muni.cz/pub/linux/fedora/epel/7/x86_64/
errata: state: FAILED error: ! 'insertDocument :: caused by :: 17280 Btree::insert: key too large to index, failing pulp_database.units_erratum.$title_1 1749 { : "globus-authz-3.9-1.el7,globus-authz-callout-error-3.4-1.el7,globus-callout-3.12-1.el7,globus-common-15.25-1.el7,globus-ftp-client-8.12-1.el7,globus-ft..." }'
*** Bug 1175616 has been marked as a duplicate of this bug. ***
I just hit this too, synching EPEL 7 to a fresh install of Katello 2.1. http://dl.fedoraproject.org/pub/epel/7/x86_64/ {"state"=>"FAILED", "error"=>"insertDocument :: caused by :: 17280 Btree::insert: key too large to index, failing pulp_database.units_erratum.$title_1 1408 { : \"globus-common-15.26-1.el7,globus-gsi-proxy-ssl-5.7-1.el7,globus-gsi-openssl-error-3.5-1.el7,globus-openssl-module-4.6-1.el7,globus-gsi-cert-utils-9.10...\" }"},