Bug 1404345

Summary: Sync failure - DocumentTooLarge
Product: Red Hat Satellite Reporter: Brad Buckingham <bbuckingham>
Component: PulpAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED ERRATA QA Contact: jcallaha
Severity: urgent Docs Contact:
Priority: urgent    
Version: 6.2.4CC: abalakht, bmbouter, daviddavis, dcaplan, dkliban, egolov, ehelms, ggainey, gpayelka, ipanova, j.bittner, jcallaha, jentrena, kabbott, mattison_ward, mhrivnak, mmccune, pcreech, pdwyer, rchan, ttereshc, vijsingh
Target Milestone: UnspecifiedKeywords: PrioBumpField, PrioBumpPM, Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: pulp-rpm-2.8.7.9-1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-05-01 13:56:59 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Patch to compress RPM metadata, compatible with pulp_rpm 2.8.7
none
Updated patch to compress RPM metadata for pulp_rpm 2.8.7
none
Patch to compress RPM metadata with re-numbered migration for pulp_rpm 2.8.7 none

Description Brad Buckingham 2016-12-13 16:03:02 UTC 
Description of problem:

Unable to sync a repository that has RPMs that are 'too big' (e.g. 1+ GB).

Version-Release number of selected component (if applicable):

Satellite 6.2

How reproducible:

Always, if the RPM is over the limit

Steps to Reproduce:

1. create a repo (e.g. https://fedorapeople.org/groups/katello/releases/yum/foreman/1.13/el6/x86_64/)
2. sync the repo

Actual results:

The sync will fail with an error similar to the following:

Dec 13 14:43:42 bye-capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (31174-60384) command document too large
Dec 13 14:43:42 bye-capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (31174-60384) Traceback (most recent call last):
Dec 13 14:43:42 bye-capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (31174-60384)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 224, in run
Dec 13 14:43:42 bye-capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (31174-60384)     self.update_content(metadata_files, url)
Dec 13 14:43:42 bye-capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (31174-60384)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 488, in update_content
Dec 13 14:43:42 bye-capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (31174-60384)     self.download_rpms(metadata_files, rpms_to_download, url)
Dec 13 14:43:42 bye-capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (31174-60384)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 685, in download_rpms
Dec 13 14:43:42 bye-capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (31174-60384)     unit = self.add_rpm_unit(metadata_files, unit)
Dec 13 14:43:42 bye-capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (31174-60384)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 632, in add_rpm_unit
Dec 13 14:43:42 bye-capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (31174-60384)     unit.save()
Dec 13 14:43:42 bye-capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (31174-60384)   File "/usr/lib/python2.7/site-packages/mongoengine/document.py", line 324, in save
Dec 13 14:43:42 bye-capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (31174-60384)     object_id = collection.save(doc, **write_concern)
Dec 13 14:43:42 bye-capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (31174-60384)   File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 2180, in save
Dec 13 14:43:42 bye-capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (31174-60384)     check_keys, False, manipulate, write_concern)
Dec 13 14:43:42 bye-capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (31174-60384)   File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 709, in _update
Dec 13 14:43:42 bye-capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (31174-60384)     codec_options=self.codec_options).copy()
Dec 13 14:43:42 bye-capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (31174-60384)   File "/usr/lib64/python2.7/site-packages/pymongo/pool.py", line 216, in command
Dec 13 14:43:42 bye-capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (31174-60384)     self._raise_connection_failure(error)
Dec 13 14:43:42 bye-capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (31174-60384)   File "/usr/lib64/python2.7/site-packages/pymongo/pool.py", line 343, in _raise_connection_failure
Dec 13 14:43:42 bye-capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (31174-60384)     raise error
Dec 13 14:43:42 bye-capsule pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (31174-60384) DocumentTooLarge: command document too large
Dec 13 14:43:42 bye-capsule pulp: py.warnings:WARNING: (31174-60384) /usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py:804: DeprecationWarning: save is deprecated. Use insert_one or replace_one instead
Dec 13 14:43:42 bye-capsule pulp: py.warnings:WARNING: (31174-60384)   sync_result_collection.save(sync_result)
Dec 13 14:43:42 bye-capsule pulp: py.warnings:WARNING: (31174-60384)
Dec 13 14:43:42 bye-capsule pulp: pulp.server.async.tasks:INFO: Task failed : [a9d888c3-8886-44b6-abf2-64141d6912d5]
Dec 13 14:43:43 bye-capsule pulp: celery.worker.job:ERROR: (30789-60384) Task pulp.server.managers.repo.sync.sync[a9d888c3-8886-44b6-abf2-64141d6912d5] raised unexpected: PulpExecutionException('Importer indicated a failed response',)
Dec 13 14:43:43 bye-capsule pulp: celery.worker.job:ERROR: (30789-60384) Traceback (most recent call last):
Dec 13 14:43:43 bye-capsule pulp: celery.worker.job:ERROR: (30789-60384)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Dec 13 14:43:43 bye-capsule pulp: celery.worker.job:ERROR: (30789-60384)     R = retval = fun(*args, **kwargs)
Dec 13 14:43:43 bye-capsule pulp: celery.worker.job:ERROR: (30789-60384)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 488, in __call__
Dec 13 14:43:43 bye-capsule pulp: celery.worker.job:ERROR: (30789-60384)     return super(Task, self).__call__(*args, **kwargs)
Dec 13 14:43:43 bye-capsule pulp: celery.worker.job:ERROR: (30789-60384)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in __call__
Dec 13 14:43:43 bye-capsule pulp: celery.worker.job:ERROR: (30789-60384)     return super(PulpTask, self).__call__(*args, **kwargs)
Dec 13 14:43:43 bye-capsule pulp: celery.worker.job:ERROR: (30789-60384)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
Dec 13 14:43:43 bye-capsule pulp: celery.worker.job:ERROR: (30789-60384)     return self.run(*args, **kwargs)
Dec 13 14:43:43 bye-capsule pulp: celery.worker.job:ERROR: (30789-60384)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 810, in sync
Dec 13 14:43:43 bye-capsule pulp: celery.worker.job:ERROR: (30789-60384)     raise pulp_exceptions.PulpExecutionException(_('Importer indicated a failed response'))
Dec 13 14:43:43 bye-capsule pulp: celery.worker.job:ERROR: (30789-60384) PulpExecutionException: Importer indicated a failed response
Dec 13 14:43:43 bye-capsule pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[83065288-0fc5-4f56-9dc0-d3089e8b5162] succeeded in 0.0121840410002s: None
Dec 13 14:44:34 bye-capsule pulp: celery.beat:INFO: Scheduler: Sending due task download_deferred_content (pulp.server.controllers.repository.queue_download_deferred)
Dec 13 14:44:34 bye-capsule pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.queue_download_deferred[659a3d93-4df1-47cd-bc69-9938359aa351]
Dec 13 14:44:34 bye-capsule pulp: celery.worker.strategy:INFO: Received task: pulp.server.controllers.repository.download_deferred[5974d77b-bfad-4b55-9b25-b3d9501f5c1c]
Dec 13 14:44:34 bye-capsule pulp: celery.worker.job:INFO: Task pulp.server.controllers.repository.queue_download_deferred[659a3d93-4df1-47cd-bc69-9938359aa351] succeeded in 0.041977586s: None
Dec 13 14:44:34 bye-capsule pulp: celery.worker.job:INFO: Task pulp.server.controllers.repository.download_deferred[5974d77b-bfad-4b55-9b25-b3d9501f5c1c] succeeded in 0.040159016s: None

Expected results:

No error and the content is synced.

Additional info:
Comment 4 pulp-infra@redhat.com 2016-12-13 17:44:11 UTC 
The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.
Comment 5 pulp-infra@redhat.com 2016-12-13 17:44:13 UTC 
The Pulp upstream bug priority is at High. Updating the external tracker on this bug.
Comment 6 pulp-infra@redhat.com 2016-12-16 00:50:56 UTC 
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.
Comment 7 pulp-infra@redhat.com 2017-01-20 02:04:30 UTC 
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.
Comment 10 Tanya Tereshchenko 2017-02-27 17:52:58 UTC 
Created attachment 1258173 [details]
Patch to compress RPM metadata, compatible with pulp_rpm 2.8.7
Comment 11 Tanya Tereshchenko 2017-02-27 20:28:40 UTC 
Created attachment 1258189 [details]
Updated patch to compress RPM metadata for pulp_rpm 2.8.7
Comment 12 pulp-infra@redhat.com 2017-02-28 23:05:03 UTC 
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.
Comment 13 pulp-infra@redhat.com 2017-02-28 23:34:49 UTC 
All upstream Pulp bugs are at MODIFIED+. Moving this bug to POST.
Comment 17 Tanya Tereshchenko 2017-03-13 22:48:43 UTC 
Created attachment 1262699 [details]
Patch to compress RPM metadata with re-numbered migration for pulp_rpm 2.8.7
Comment 19 Dennis Kliban 2017-03-14 02:31:24 UTC 
The Pulp upstream bug status is at ON_QA. Updating the external tracker on this bug.
Comment 23 pulp-infra@redhat.com 2017-03-30 02:06:53 UTC 
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.
Comment 24 pulp-infra@redhat.com 2017-03-30 02:06:59 UTC 
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.
Comment 25 pulp-infra@redhat.com 2017-03-30 23:06:53 UTC 
The Pulp upstream bug status is at CLOSED - COMPLETE. Updating the external tracker on this bug.
Comment 26 jcallaha 2017-04-03 18:08:22 UTC 
Verified in Satellite 6.2.9 Snap 2

Followed the steps outlined in the bz description. The repository sync'd correctly, both when in on_demand and then in immediate download modes. See attached screenshot for verification.
Comment 27 Mike McCune 2017-04-06 19:24:49 UTC 
=== HOTFIX INSTRUCTIONS FOR SATELLITE 6.2.8 ONLY ===


See the hotfix package and instructions outlined here:

https://bugzilla.redhat.com/show_bug.cgi?id=1388296#c49
Comment 28 pulp-infra@redhat.com 2017-04-19 21:37:22 UTC 
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.
Comment 30 errata-xmlrpc 2017-05-01 13:56:59 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1191