Bug 1734555 - Sync failure - DocumentTooLarge: command document too large
Summary: Sync failure - DocumentTooLarge: command document too large
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Pulp
Version: 6.5.0
Hardware: Unspecified
OS: Linux
unspecified
medium
Target Milestone: Unspecified
Assignee: satellite6-bugs
QA Contact: Kersom
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-07-30 20:56 UTC by Gourav Padholia
Modified: 2020-05-01 13:55 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-05-01 13:55:31 UTC
Target Upstream Version:


Attachments (Terms of Use)

Comment 3 Gourav Padholia 2019-07-31 13:55:43 UTC
Description of problem:
Unable to Sync a package (RPM) which has lots of files. 

Version-Release number of selected component (if applicable):
Satellite 6.4 and Satellite 6.5

How reproducible:

100 %
Steps to Reproduce:
1. Create a rpm package with lots of file ( more than 100K ).
2. Create a custom product from Satellite web UI and create customer repository under that package. 
3. Upload the package using "hammer repository upload-content" command.

Actual results:
The sync will fail with an error similar to the following in /var/log/messages :

Jul 31 02:32:35 satellite64 pulp: pulp_rpm.plugins.importers.yum.upload:ERROR: [8ce11872] (24085-90304) unexpected error occurred importing uploaded file: command document too large
Jul 31 02:32:35 satellite64 pulp: pulp_rpm.plugins.importers.yum.upload:ERROR: [8ce11872] (24085-90304) Traceback (most recent call last):
Jul 31 02:32:35 satellite64 pulp: pulp_rpm.plugins.importers.yum.upload:ERROR: [8ce11872] (24085-90304)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/upload.py", line 120, in upload
Jul 31 02:32:35 satellite64 pulp: pulp_rpm.plugins.importers.yum.upload:ERROR: [8ce11872] (24085-90304)     handlers[type_id](repo, type_id, unit_key, metadata, file_path, conduit, config)
Jul 31 02:32:35 satellite64 pulp: pulp_rpm.plugins.importers.yum.upload:ERROR: [8ce11872] (24085-90304)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/upload.py", line 440, in _handle_package
Jul 31 02:32:35 satellite64 pulp: pulp_rpm.plugins.importers.yum.upload:ERROR: [8ce11872] (24085-90304)     unit.save_and_import_content(file_path)
Jul 31 02:32:35 satellite64 pulp: pulp_rpm.plugins.importers.yum.upload:ERROR: [8ce11872] (24085-90304)   File "/usr/lib/python2.7/site-packages/pulp/server/db/model/__init__.py", line 918, in save_and_import_content
Jul 31 02:32:35 satellite64 pulp: pulp_rpm.plugins.importers.yum.upload:ERROR: [8ce11872] (24085-90304)     self.save()
Jul 31 02:32:35 satellite64 pulp: pulp_rpm.plugins.importers.yum.upload:ERROR: [8ce11872] (24085-90304)   File "/usr/lib/python2.7/site-packages/mongoengine/document.py", line 324, in save
Jul 31 02:32:35 satellite64 pulp: pulp_rpm.plugins.importers.yum.upload:ERROR: [8ce11872] (24085-90304)     object_id = collection.save(doc, **write_concern)
Jul 31 02:32:35 satellite64 pulp: pulp_rpm.plugins.importers.yum.upload:ERROR: [8ce11872] (24085-90304)   File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 2180, in save
Jul 31 02:32:35 satellite64 pulp: pulp_rpm.plugins.importers.yum.upload:ERROR: [8ce11872] (24085-90304)     check_keys, False, manipulate, write_concern)
Jul 31 02:32:35 satellite64 pulp: pulp_rpm.plugins.importers.yum.upload:ERROR: [8ce11872] (24085-90304)   File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 709, in _update
Jul 31 02:32:35 satellite64 pulp: pulp_rpm.plugins.importers.yum.upload:ERROR: [8ce11872] (24085-90304)     codec_options=self.codec_options).copy()
Jul 31 02:32:35 satellite64 pulp: pulp_rpm.plugins.importers.yum.upload:ERROR: [8ce11872] (24085-90304)   File "/usr/lib64/python2.7/site-packages/pymongo/pool.py", line 216, in command
Jul 31 02:32:35 satellite64 pulp: pulp_rpm.plugins.importers.yum.upload:ERROR: [8ce11872] (24085-90304)     self._raise_connection_failure(error)
Jul 31 02:32:35 satellite64 pulp: pulp_rpm.plugins.importers.yum.upload:ERROR: [8ce11872] (24085-90304)   File "/usr/lib64/python2.7/site-packages/pymongo/pool.py", line 343, in _raise_connection_failure
Jul 31 02:32:35 satellite64 pulp: pulp_rpm.plugins.importers.yum.upload:ERROR: [8ce11872] (24085-90304)     raise error
Jul 31 02:32:35 satellite64 pulp: pulp_rpm.plugins.importers.yum.upload:ERROR: [8ce11872] (24085-90304) DocumentTooLarge: command document too large


Expected results:
No error and rpm should be synced without successfully.

Comment 8 Tanya Tereshchenko 2020-05-01 13:55:31 UTC
Pulp 2 is in maintenance mode and currently accepts only critical/security issues. The main focus is on Pulp 3 and some of the requests will be satisfied in the newer version.
We have evaluated this request, and while we recognize that it is a valid request, we do not expect this to be implemented in Pulp 2. As this issue is not relevant for Pulp 3, we are therefore closing this out as WONTFIX.


Note You need to log in before you can comment on or make changes to this bug.