Description of problem: Prior to and after upgrading from 2.5.0 to 2.5.1 (and now also after applying the 2.5.2-0.0.beta packages), I've been unable to publish both feed-url and upload rpm repos. The issue appears to affect all of my repos. Version-Release number of selected component (if applicable): 2.5.0 2.5.1 2.5.2-0.0.beta How reproducible: Always Steps to Reproduce: 1. pulp-admin rpm repo publish run --repo-id=<repo name> OR pulp-admin rpm repo sync run --repo-id=<repo-name> 2. 3. Actual results: # pulp-admin rpm repo publish run --repo-id=rhel-6-x86_64-os +----------------------------------------------------------------------+ Publishing Repository [rhel-6-x86_64-os] +----------------------------------------------------------------------+ This command may be exited via ctrl+c without affecting the request. Initializing repo metadata [-] ... completed Publishing Distribution files [-] ... completed Publishing RPMs [====================== ] 45% 6605 of 14365 items ... failed Incorrect length of data produced Task Failed Incorrect length of data produced # pulp-admin rpm repo sync run --repo-id=rhel-6-x86_64-os +----------------------------------------------------------------------+ Synchronizing Repository [rhel-6-x86_64-os] +----------------------------------------------------------------------+ This command may be exited via ctrl+c without affecting the request. Downloading metadata... [\] ... completed Downloading repository content... [==================================================] 100% RPMs: 113/113 items Delta RPMs: 0/0 items ... completed Downloading distribution files... [==================================================] 100% Distributions: 0/0 items ... completed Importing errata... [-] ... completed Importing package groups/categories... [\] ... completed Task Succeeded Initializing repo metadata [-] ... completed Publishing Distribution files [-] ... completed Publishing RPMs [====================== ] 45% 6643 of 14478 items ... failed Incorrect length of data produced Task Failed Incorrect length of data produced Expected results: Successfully publishing repos for use via yum. Additional info: Current package version (although this was failing prior to upgrading to this version): yum list installed |grep pulp grinder.noarch 0.1.16-1.el6 @pulp-v2-stable m2crypto.x86_64 0.21.1.pulp-8.el6 @pulp-v2-stable mod_wsgi.x86_64 3.4-2.pulp.el6 @pulp-v2-stable pulp-admin-client.noarch 2.5.2-0.0.beta.el6 @pulp-v2-beta pulp-puppet-admin-extensions.noarch 2.5.2-0.0.beta.el6 @pulp-v2-beta pulp-puppet-plugins.noarch 2.5.2-0.0.beta.el6 @pulp-v2-beta pulp-rpm-admin-extensions.noarch 2.5.2-0.0.beta.el6 @pulp-v2-beta pulp-rpm-plugins.noarch 2.5.2-0.0.beta.el6 @pulp-v2-beta pulp-selinux.noarch 2.5.2-0.0.beta.el6 @pulp-v2-beta pulp-server.noarch 2.5.2-0.0.beta.el6 @pulp-v2-beta python-amqp.noarch 1.4.5-1.el6 @pulp-v2-stable python-anyjson.noarch 0.3.3-4.el6 @pulp-v2-stable python-billiard.x86_64 1:3.3.0.17-1.el6 @pulp-v2-stable python-celery.noarch 3.1.11-1.el6 @pulp-v2-stable python-gofer.noarch 1.3.1-1.el6 @pulp-v2-stable python-gofer-qpid.noarch 1.3.1-1.el6 @pulp-v2-stable python-isodate.noarch 0.5.0-4.pulp.el6 @pulp-v2-stable python-kombu.noarch 1:3.0.15-12.pulp.el6 @pulp-v2-stable python-nectar.noarch 1.3.1-1.el6 @pulp-v2-stable python-pulp-bindings.noarch 2.5.2-0.0.beta.el6 @pulp-v2-beta python-pulp-client-lib.noarch 2.5.2-0.0.beta.el6 @pulp-v2-beta python-pulp-common.noarch 2.5.2-0.0.beta.el6 @pulp-v2-beta python-pulp-puppet-common.noarch 2.5.2-0.0.beta.el6 @pulp-v2-beta python-pulp-rpm-common.noarch 2.5.2-0.0.beta.el6 @pulp-v2-beta python-qpid.noarch 0.26-2.el6 @pulp-v2-stable python-qpid-common.noarch 0.26-2.el6 @pulp-v2-stable python-qpid-qmf.x86_64 0.26-2.el6 @pulp-v2-stable python-requests.noarch 2.2.1-1.el6 @pulp-v2-stable python-semantic-version.noarch 2.2.0-3.el6 @pulp-v2-stable qpid-cpp-client.x86_64 0.26-9.el6 @pulp-v2-beta qpid-cpp-server.x86_64 0.26-9.el6 @pulp-v2-beta qpid-cpp-server-store.x86_64 0.26-9.el6 @pulp-v2-beta qpid-qmf.x86_64 0.26-2.el6 @pulp-v2-stable Trace from /var/log/messages: Dec 22 14:57:51 dvpuap02 pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._reserve_resource[831267ce-98eb-41c3-9dad-0b0e32035ec0] Dec 22 14:57:51 dvpuap02 pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._reserve_resource[831267ce-98eb-41c3-9dad-0b0e32035ec0] succeeded in 0.031 6150639992s: u'reserved_resource_worker-2.edu.dq' Dec 22 14:57:52 dvpuap02 pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.publish.publish[7003450d-2b88-45a5-b05b-f5a92cfbba67] Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: Exception caught from plugin during publish for repo [epel6] Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: Traceback (most recent call last): Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py", line 1 26, in _do_publish Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: publish_report = publish_repo(transfer_repo, conduit, call_config) Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 470, in wra p_f Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: return f(*args, **kwargs) Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/distributor .py", line 143, in publish_repo Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: return self._publisher.publish() Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 507, in publish Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: return self.process_lifecycle() Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 456, in process_lifecycle Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: super(PluginStep, self).process_lifecycle() Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 121, in process_lifecycle Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: step.process() Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 185, in process Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: self.finalize() Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 414, in finalize Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: self.other_context.checksum) Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/metadata/repomd.py", line 115, in add_metadata_file_metadata Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: content = file_handle.read() Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib64/python2.6/gzip.py", line 212, in read Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: self._read(readsize) Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib64/python2.6/gzip.py", line 284, in _read Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: self._read_eof() Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib64/python2.6/gzip.py", line 306, in _read_eof Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: raise IOError, "Incorrect length of data produced" Dec 22 14:57:58 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: IOError: Incorrect length of data produced Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: Task pulp.server.managers.repo.publish.publish[7003450d-2b88-45a5-b05b-f5a92cfbba67] raised unexpected: IOError('Incorrect length of data produced',) Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: Traceback (most recent call last): Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 240, in trace_task Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: R = retval = fun(*args, **kwargs) Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 315, in __call__ Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: return super(Task, self).__call__(*args, **kwargs) Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in __protected_call__ Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: return self.run(*args, **kwargs) Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py", line 98, in publish Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: transfer_repo, conduit, call_config) Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py", line 126, in _do_publish Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: publish_report = publish_repo(transfer_repo, conduit, call_config) Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 470, in wrap_f Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: return f(*args, **kwargs) Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 143, in publish_repo Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: return self._publisher.publish() Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 507, in publish Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: return self.process_lifecycle() Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 456, in process_lifecycle Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: super(PluginStep, self).process_lifecycle() Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 121, in process_lifecycle Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: step.process() Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 185, in process Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: self.finalize() Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 414, in finalize Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: self.other_context.checksum) Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/metadata/repomd.py", line 115, in add_metadata_file_metadata Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: content = file_handle.read() Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib64/python2.6/gzip.py", line 212, in read Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: self._read(readsize) Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib64/python2.6/gzip.py", line 284, in _read Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: self._read_eof() Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib64/python2.6/gzip.py", line 306, in _read_eof Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: raise IOError, "Incorrect length of data produced" Dec 22 14:57:58 dvpuap02 pulp: celery.worker.job:ERROR: IOError: Incorrect length of data produced
Please also share the traceback that occurs during sync. Is this reproducible by syncing a publicly-available repository? Let us know which so we can try to reproduce it here.
For reference, this is the call in pulp's publish workflow that results in an exception from python's gzip module: https://github.com/pulp/pulp_rpm/blob/pulp-rpm-2.5.2-0.0.beta/plugins/pulp_rpm/plugins/distributors/yum/metadata/repomd.py#L115
*** Bug 1177627 has been marked as a duplicate of this bug. ***
The publish error occurs on several of my 17 repos. I thought it was all of them as the first 4 I attempted all failed. The following are failing: https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/os https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/optional/os http://public-yum.oracle.com/repo/OracleLinux/OL5/latest/x86_64 https://cdn.redhat.com/content/dist/rhel/server/5/5Server/x86_64/os http://mirror.us.leaseweb.net/epel/6/x86_64/ http://mirror.us.leaseweb.net/epel/5/x86_64/ These are publishing correctly: https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/rhscl/1/os https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/debug https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/optional/os https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/rhscl/1/os https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/debug http://dl.iuscommunity.org/pub/ius/stable/Redhat/6/x86_64/ http://mirror.pnl.gov/epel/7/x86_64/ https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/scalablefilesystem/os http://public-yum.oracle.com/repo/OracleLinux/OL6/latest/x86_64 https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os A couple of items of note: 1. The publish failures always happen on the same package for each repo (e.g. for epel6 at 75 of 19440 items, for rhel6-optional at 54 of 8104 items) 2. I have a repo called rhel-6-x86_64-os-physical which is a point in time copy of the rhel6 os repo just prior to 6.6 being released. That repo is publishing fine but the rhel6 os repo is not. Here is the traceback from the sync of the rhel6 os repo: Jan 2 11:48:06 dvpuap02 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading 1 RPMs. Jan 2 11:51:06 dvpuap02 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com Jan 2 11:52:33 dvpuap02 pulp: pulp_rpm.plugins.importers.yum.repomd.alternate:INFO: The content container reported: {'downloads': {'___/primary/___': {'tot al_failed': 0, 'total_succeeded': 1}}, 'total_sources': 0} for base URL: https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/os/ Jan 2 11:52:33 dvpuap02 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading additional units. Jan 2 11:52:33 dvpuap02 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com Jan 2 11:52:34 dvpuap02 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com Jan 2 11:52:49 dvpuap02 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Sync complete. Jan 2 11:52:49 dvpuap02 pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._reserve_resource[fc8b7a5f-5911-43ae-ad11-4e2661791ede] Jan 2 11:52:49 dvpuap02 pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._reserve_resource[fc8b7a5f-5911-43ae-ad11-4e2661791ede] succeeded in 0.0 31124184s: u'reserved_resource_worker-0.edu.dq' Jan 2 11:52:50 dvpuap02 pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.publish.publish[f0a6ad0e-2fe6-4eaa-838c-465b91db9d37] Jan 2 11:52:50 dvpuap02 pulp: celery.worker.job:INFO: Task pulp.server.tasks.repository.sync_with_auto_publish[4b8189f1-6c87-40a1-97f2-3176bab09b76] succeeded in 1773.61550857s: <pulp.server.async.tasks.TaskResult object at 0x7fb75803eb50> Jan 2 11:52:58 dvpuap02 rsyslogd-2177: imuxsock begins to drop messages from pid 2344 due to rate-limiting Jan 2 11:53:09 dvpuap02 rsyslogd-2177: imuxsock lost 352 messages from pid 2344 due to rate-limiting Jan 2 11:53:09 dvpuap02 rsyslogd-2177: imuxsock begins to drop messages from pid 2344 due to rate-limiting Jan 2 11:53:15 dvpuap02 rsyslogd-2177: imuxsock lost 130 messages from pid 2344 due to rate-limiting Jan 2 11:53:17 dvpuap02 rsyslogd-2177: imuxsock begins to drop messages from pid 2344 due to rate-limiting Jan 2 11:53:21 dvpuap02 rsyslogd-2177: imuxsock lost 872 messages from pid 2344 due to rate-limiting Jan 2 11:53:22 dvpuap02 rsyslogd-2177: imuxsock begins to drop messages from pid 2344 due to rate-limiting Jan 2 11:53:27 dvpuap02 rsyslogd-2177: imuxsock lost 853 messages from pid 2344 due to rate-limiting Jan 2 11:53:27 dvpuap02 rsyslogd-2177: imuxsock begins to drop messages from pid 2344 due to rate-limiting Jan 2 11:53:33 dvpuap02 rsyslogd-2177: imuxsock lost 974 messages from pid 2344 due to rate-limiting Jan 2 11:53:42 dvpuap02 rsyslogd-2177: imuxsock begins to drop messages from pid 2344 due to rate-limiting Jan 2 11:53:45 dvpuap02 rsyslogd-2177: imuxsock lost 68 messages from pid 2344 due to rate-limiting Jan 2 11:54:44 dvpuap02 rsyslogd-2177: imuxsock begins to drop messages from pid 2344 due to rate-limiting Jan 2 11:54:51 dvpuap02 rsyslogd-2177: imuxsock lost 50 messages from pid 2344 due to rate-limiting Jan 2 11:54:52 dvpuap02 rsyslogd-2177: imuxsock begins to drop messages from pid 2344 due to rate-limiting Jan 2 11:54:59 dvpuap02 rsyslogd-2177: imuxsock lost 78 messages from pid 2344 due to rate-limiting Jan 2 11:55:00 dvpuap02 rsyslogd-2177: imuxsock begins to drop messages from pid 2344 due to rate-limiting Jan 2 11:55:05 dvpuap02 rsyslogd-2177: imuxsock lost 481 messages from pid 2344 due to rate-limiting Jan 2 11:55:08 dvpuap02 rsyslogd-2177: imuxsock begins to drop messages from pid 2344 due to rate-limiting Jan 2 11:55:14 dvpuap02 rsyslogd-2177: imuxsock lost 172 messages from pid 2344 due to rate-limiting Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: Exception caught from plugin during publish for repo [rhel-6-x86_64-os] Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: Traceback (most recent call last): Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py", line 126, in _do_publish Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: publish_report = publish_repo(transfer_repo, conduit, call_config) Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 470, in wrap_f Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: return f(*args, **kwargs) Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 143, in publish_repo Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: return self._publisher.publish() Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 507, in publish Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: return self.process_lifecycle() Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 456, in process_lifecycle Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: super(PluginStep, self).process_lifecycle() Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 121, in process_lifecycle Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: step.process() Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 185, in process Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: self.finalize() Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 414, in finalize Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: self.other_context.checksum) Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/metadata/repomd.py", line 115, in add_metadata_file_metadata Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: content = file_handle.read() Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib64/python2.6/gzip.py", line 212, in read Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: self._read(readsize) Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib64/python2.6/gzip.py", line 284, in _read Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: self._read_eof() Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib64/python2.6/gzip.py", line 306, in _read_eof Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: raise IOError, "Incorrect length of data produced" Jan 2 11:55:22 dvpuap02 pulp: pulp.server.managers.repo.publish:ERROR: IOError: Incorrect length of data produced Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: Task pulp.server.managers.repo.publish.publish[f0a6ad0e-2fe6-4eaa-838c-465b91db9d37] raised unexpected: IOError('Incorrect length of data produced',) Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: Traceback (most recent call last): Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 240, in trace_task Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: R = retval = fun(*args, **kwargs) Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 315, in __call__ Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: return super(Task, self).__call__(*args, **kwargs) Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in __protected_call__ Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: return self.run(*args, **kwargs) Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py", line 98, in publish Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: transfer_repo, conduit, call_config) Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py", line 126, in _do_publish Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: publish_report = publish_repo(transfer_repo, conduit, call_config) Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 470, in wrap_f Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: return f(*args, **kwargs) Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 143, in publish_repo Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: return self._publisher.publish() Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 507, in publish Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: return self.process_lifecycle() Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 456, in process_lifecycle Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: super(PluginStep, self).process_lifecycle() Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 121, in process_lifecycle Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: step.process() Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 185, in process Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: self.finalize() Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 414, in finalize Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: self.other_context.checksum) Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/metadata/repomd.py", line 115, in add_metadata_file_metadata Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: content = file_handle.read() Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib64/python2.6/gzip.py", line 212, in read Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: self._read(readsize) Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib64/python2.6/gzip.py", line 284, in _read Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: self._read_eof() Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: File "/usr/lib64/python2.6/gzip.py", line 306, in _read_eof Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: raise IOError, "Incorrect length of data produced" Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:ERROR: IOError: Incorrect length of data produced Jan 2 11:55:22 dvpuap02 pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[8c30c2b7-e142-4550-9914-dc1145cbd8f8] Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_release_resource[0e332bf8-5a6a-4331-b6f7-66c24b85f6a0] succeeded in 0.0150966489996s: None Jan 2 11:55:22 dvpuap02 pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[8c30c2b7-e142-4550-9914-dc1145cbd8f8] succeeded in 0.042722529s: None
I think this issue it's related to the bug #1158945. The proposed patch (https://github.com/pulp/pulp_rpm/pull/605) fixes the problem but on CentOS 6.5 you have to modify the patch to use encode('utf-8') instead of encode(encoding='utf-8').
I was able sync and publish rhel6 repo from cdn. Pasting the results. Please let us know if we are missing something. [root@ibm-x3250m4-02 ~]# pulp-admin rpm repo create --repo-id rhel6 --feed https://cdn.redhat.com/content/dist/rhel/rhui/server/6/6.6/x86_64/os/ --feed-ca-cert CDN/cdn.redhat.com-chain.crt --feed-cert CDN/1391032266_4771.crt --feed-key CDN/1391032266_4771.key Successfully created repository [rhel6] [root@ibm-x3250m4-02 ~]# [root@ibm-x3250m4-02 ~]# [root@ibm-x3250m4-02 ~]# pulp-admin rpm repo sync run --repo-id rhel6 +----------------------------------------------------------------------+ Synchronizing Repository [rhel6] +----------------------------------------------------------------------+ This command may be exited via ctrl+c without affecting the request. Downloading metadata... [|] ... completed Downloading repository content... [==================================================] 100% RPMs: 14472/14472 items Delta RPMs: 0/0 items ... completed Downloading distribution files... [==================================================] 100% Distributions: 0/0 items ... completed Importing errata... [\] ... completed Importing package groups/categories... [|] ... completed Task Succeeded Initializing repo metadata [-] ... completed Publishing Distribution files [-] ... completed Publishing RPMs [==================================================] 100% 14472 of 14472 items ... completed Publishing Delta RPMs ... skipped Publishing Errata [==================================================] 100% 2818 of 2818 items ... completed Publishing Comps file [==================================================] 100% 212 of 212 items ... completed Publishing Metadata. [-] ... completed Closing repo metadata [-] ... completed Generating sqlite files ... skipped Publishing files to web [|] ... completed Writing Listings File [-] ... completed Task Succeeded [root@ibm-x3250m4-02 ~]#
https://github.com/pulp/pulp_rpm/pull/623
This was merged to 2.6-dev which aligns to 2.6.1. If you need this fix in 2.6.0, please create a PR on 2.6-testing, and set the bug's target release back to 2.6.0. Setting target release to 2.6.1.
>> rpm -qa pulp-server pulp-server-2.6.0-0.4.beta.el6.noarch >> pulp-admin rpm repo create --repo-id epel6_1 --feed http://mirror.karneval.cz/pub/linux/fedora/epel/6/x86_64/ Successfully created repository [epel6_1] >> pulp-admin rpm repo sync run --repo-id epel6_1 +----------------------------------------------------------------------+ Synchronizing Repository [epel6_1] +----------------------------------------------------------------------+ This command may be exited via ctrl+c without affecting the request. Downloading metadata... [|] ... completed Downloading repository content... [==================================================] 100% RPMs: 11178/11178 items Delta RPMs: 0/0 items ... completed Downloading distribution files... [==================================================] 100% Distributions: 0/0 items ... completed Importing errata... [\] ... completed Importing package groups/categories... [|] ... completed Task Succeeded Initializing repo metadata [-] ... completed Publishing Distribution files [-] ... completed Publishing RPMs [ ] 0% 1 of 11178 items ... failed encode() takes no keyword arguments Task Failed encode() takes no keyword arguments
pulp 2.6.0 beta 5
after update to pulp-server-2.6.0-0.5.beta.el6.noarch >> pulp-admin rpm repo sync run --repo-id epel6_1 +----------------------------------------------------------------------+ Synchronizing Repository [epel6_1] +----------------------------------------------------------------------+ This command may be exited via ctrl+c without affecting the request. Downloading metadata... [\] ... completed Downloading repository content... [==================================================] 100% RPMs: 1/1 items Delta RPMs: 0/0 items ... completed Downloading distribution files... [==================================================] 100% Distributions: 0/0 items ... completed Importing errata... [-] ... completed Importing package groups/categories... [|] ... completed Task Succeeded Initializing repo metadata [-] ... completed Publishing Distribution files [-] ... completed Publishing RPMs [==================================================] 100% 11178 of 11178 items ... completed Publishing Delta RPMs ... skipped Publishing Errata [==================================================] 100% 3635 of 3635 items ... completed Publishing Comps file [==================================================] 100% 211 of 211 items ... completed Publishing Metadata. [-] ... completed Closing repo metadata [-] ... completed Generating sqlite files ... skipped Publishing files to web [|] ... completed Writing Listings File [-] ... completed Task Succeeded >> pulp-admin rpm repo publish run --repo-id epel6_1 +----------------------------------------------------------------------+ Publishing Repository [epel6_1] +----------------------------------------------------------------------+ This command may be exited via ctrl+c without affecting the request. Copying files [|] ... completed Initializing repo metadata [-] ... completed Publishing Distribution files [-] ... completed Publishing RPMs [/] ... completed Publishing Delta RPMs ... skipped Publishing Errata [==================================================] 100% 3635 of 3635 items ... completed Publishing Comps file [==================================================] 100% 211 of 211 items ... completed Publishing Metadata. [\] ... completed Closing repo metadata [-] ... completed Generating sqlite files ... skipped Publishing files to web [/] ... completed Writing Listings File [-] ... completed Task Succeeded >> pulp-admin rpm repo list +----------------------------------------------------------------------+ RPM Repositories +----------------------------------------------------------------------+ Id: epel6_1 Display Name: epel6_1 Description: None Content Unit Counts: Erratum: 3635 Package Category: 3 Package Group: 208 Rpm: 11178 Yum Repo Metadata File: 1 >> ls -l /var/lib/pulp/published/yum/master/yum_distributor/ total 4 drwxr-xr-x. 3 apache apache 4096 Jan 16 22:51 epel6_1 I also checked this http://mirror.us.leaseweb.net/epel/5/x86_64/ from commnet#4 >> pulp-admin rpm repo create --repo-id epel5 --feed http://mirror.us.leaseweb.net/epel/5/x86_64/ Successfully created repository [epel5] >> pulp-admin rpm repo sync run --repo-id epel5 +----------------------------------------------------------------------+ Synchronizing Repository [epel5] +----------------------------------------------------------------------+ This command may be exited via ctrl+c without affecting the request. Downloading metadata... [/] ... completed Downloading repository content... [==================================================] 100% RPMs: 6848/6848 items Delta RPMs: 0/0 items ... completed Downloading distribution files... [==================================================] 100% Distributions: 0/0 items ... completed Importing errata... [-] ... completed Importing package groups/categories... [\] ... completed Task Succeeded Initializing repo metadata [-] ... completed Publishing Distribution files [-] ... completed Publishing RPMs [==================================================] 100% 6848 of 6848 items ... completed Publishing Delta RPMs ... skipped Publishing Errata [==================================================] 100% 1998 of 1998 items ... completed Publishing Comps file [==================================================] 100% 41 of 41 items ... completed Publishing Metadata. [-] ... completed Closing repo metadata [-] ... completed Generating sqlite files ... skipped Publishing files to web [\] ... completed Writing Listings File [-] ... completed Task Succeeded >> pulp-admin rpm repo publish run --repo-id epel5 +----------------------------------------------------------------------+ Publishing Repository [epel5] +----------------------------------------------------------------------+ This command may be exited via ctrl+c without affecting the request. Copying files [\] ... completed Initializing repo metadata [-] ... completed Publishing Distribution files [-] ... completed Publishing RPMs [\] ... completed Publishing Delta RPMs ... skipped Publishing Errata [==================================================] 100% 1998 of 1998 items ... completed Publishing Comps file [==================================================] 100% 41 of 41 items ... completed Publishing Metadata. [-] ... completed Closing repo metadata [-] ... completed Generating sqlite files ... skipped Publishing files to web [|] ... completed Writing Listings File [-] ... completed Task Succeeded >> pulp-admin rpm repo list +----------------------------------------------------------------------+ RPM Repositories +----------------------------------------------------------------------+ Id: epel6_1 Display Name: epel6_1 Description: None Content Unit Counts: Erratum: 3635 Package Category: 3 Package Group: 208 Rpm: 11178 Yum Repo Metadata File: 1 Id: epel5 Display Name: epel5 Description: None Content Unit Counts: Erratum: 1998 Package Category: 5 Package Group: 36 Rpm: 6848 Yum Repo Metadata File: 1 >> ls -l /var/lib/pulp/published/yum/master/yum_distributor/ total 8 drwxr-xr-x. 3 apache apache 4096 Jan 16 23:11 epel5 drwxr-xr-x. 3 apache apache 4096 Jan 16 22:51 epel6_1
I'm having the same issue with: pulp-server-2.5.1-1.el7.noarch $ pulp-admin rpm repo publish run --repo-id=test +----------------------------------------------------------------------+ Publishing Repository [test] +----------------------------------------------------------------------+ This command may be exited via ctrl+c without affecting the request. Initializing repo metadata [-] ... completed Publishing Distribution files [-] ... completed Publishing RPMs [== ] 4% 1 of 25 items ... failed Incorrect length of data produced Task Failed Incorrect length of data produced Traceback: Jan 20 07:19:51 fc-d1pulp02 pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._reserve_resource[5492606b-de1f-4b9f-9d85-17b422d611f7] Jan 20 07:19:51 fc-d1pulp02 pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._reserve_resource[5492606b-de1f-4b9f-9d85-17b422d611f7] succeeded in 0.0307118759956s: u'reserved_resource_worker-0.follett.com.dq' Jan 20 07:19:51 fc-d1pulp02 pulp: celery.worker.strategy:INFO: Received task: pulp.server.managers.repo.publish.publish[9bb85e7f-d035-478a-86b8-61c417239d9b] Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: Exception caught from plugin during publish for repo [test] Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: Traceback (most recent call last): Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/publish.py", line 126, in _do_publish Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: publish_report = publish_repo(transfer_repo, conduit, call_config) Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 470, in wrap_f Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: return f(*args, **kwargs) Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 143, in publish_repo Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: return self._publisher.publish() Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 507, in publish Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: return self.process_lifecycle() Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 456, in process_lifecycle Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: super(PluginStep, self).process_lifecycle() Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 121, in process_lifecycle Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: step.process() Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 185, in process Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: self.finalize() Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 414, in finalize Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: self.other_context.checksum) Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/metadata/repomd.py", line 115, in add_metadata_file_metadata Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: content = file_handle.read() Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib64/python2.7/gzip.py", line 254, in read Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: self._read(readsize) Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib64/python2.7/gzip.py", line 325, in _read Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: self._read_eof() Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: File "/usr/lib64/python2.7/gzip.py", line 349, in _read_eof Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: raise IOError, "Incorrect length of data produced" Jan 20 07:19:52 fc-d1pulp02 pulp: pulp.server.managers.repo.publish:ERROR: IOError: Incorrect length of data produced Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: Task pulp.server.managers.repo.publish.publish[9bb85e7f-d035-478a-86b8-61c417239d9b] raised unexpected: IOError('Incorrect length of data produced',) Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: Traceback (most recent call last): Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: R = retval = fun(*args, **kwargs) Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 315, in __call__ Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: return super(Task, self).__call__(*args, **kwargs) Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__ Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: return self.run(*args, **kwargs) Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/publish.py", line 98, in publish Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: transfer_repo, conduit, call_config) Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/publish.py", line 126, in _do_publish Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: publish_report = publish_repo(transfer_repo, conduit, call_config) Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 470, in wrap_f Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: return f(*args, **kwargs) Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 143, in publish_repo Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: return self._publisher.publish() Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 507, in publish Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: return self.process_lifecycle() Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 456, in process_lifecycle Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: super(PluginStep, self).process_lifecycle() Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 121, in process_lifecycle Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: step.process() Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 185, in process Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: self.finalize() Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 414, in finalize Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: self.other_context.checksum) Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/metadata/repomd.py", line 115, in add_metadata_file_metadata Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: content = file_handle.read() Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib64/python2.7/gzip.py", line 254, in read Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: self._read(readsize) Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib64/python2.7/gzip.py", line 325, in _read Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: self._read_eof() Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: File "/usr/lib64/python2.7/gzip.py", line 349, in _read_eof Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: raise IOError, "Incorrect length of data produced" Jan 20 07:19:52 fc-d1pulp02 pulp: celery.worker.job:ERROR: IOError: Incorrect length of data produced Is there a proposed fix for 2.5.1?
(In reply to Josh Baird from comment #12) > I'm having the same issue with: > > pulp-server-2.5.1-1.el7.noarch > > Is there a proposed fix for 2.5.1? Yes, apply these two patches: https://github.com/jeremycline/pulp_rpm/commit/6117c1ac703f4f759bd1aaa5da9a9ca8559495af.patch https://github.com/barnabycourt/pulp_rpm/commit/5bbb8b4be838ff4f4be9549bd1986efca5c23ce6.patch
This doesn't appear to apply cleanly to 2.5.1.
(In reply to Josh Baird from comment #14) > This doesn't appear to apply cleanly to 2.5.1. Then patch it manually. You only need the "if" conditions.
I can confirm that 2.6.0-0.5.beta fixes my issue and all of my repos now publish successfully.
This is still an issues with 2.5.1 after applying patch, how do i resolve this in 2.5.1..? this is bit critical.. it only does not upload some RPM..not sure what's the reason
please ignore, it does work, tested it.
Moved to https://pulp.plan.io/issues/647