Bug 620495

Summary: Error on repo syncing large repos. InvalidDocument: document too large - B SON documents are limited to 4 MB\n"
Product: [Retired] Pulp Reporter: Preethi Thomas <pthomas>
Component: z_otherAssignee: Pradeep Kilambi <pkilambi>
Status: CLOSED CURRENTRELEASE QA Contact: Preethi Thomas <pthomas>
Severity: medium Docs Contact:
Priority: high    
Version: unspecifiedCC: ssachdev, whayutin
Target Milestone: ---Keywords: QA-Closed
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 621276 (view as bug list) Environment:
Last Closed: 2010-10-20 16:11:18 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Preethi Thomas 2010-08-02 17:19:20 UTC 
Description of problem:

I have seen this with rhel5, f12 & f13 repos

When you run pulp repo sync

InvalidDocument: document too large - BSON documents are limited to 4 MB\n"


Once you hit this pulp repo delete fails as well


[root@preethi pulp]# pulp repo delete -u admin -p admin --id=f13-1
 Deleted operation failed on Repo [ f13-1 ] 


[root@preethi ~]# tail -f /var/log/pulp/pulptools.log
2010-08-02 13:15:31,870 [WARNING][Thread-1] close_engine() @ driver.py:432 - recoverable error[attempt 135]: [Errno 111] Connection refused
2010-08-02 13:15:31,870 [WARNING][Thread-1] close_engine() @ driver.py:434 - sleeping 120 seconds
2010-08-02 13:17:31,970 [WARNING][Thread-1] connect() @ driver.py:492 - trying: localhost:5672
2010-08-02 13:17:31,971 [WARNING][Thread-1] close_engine() @ driver.py:432 - recoverable error[attempt 136]: [Errno 111] Connection refused
2010-08-02 13:17:31,971 [WARNING][Thread-1] close_engine() @ driver.py:434 - sleeping 120 seconds
2010-08-02 13:19:32,071 [WARNING][Thread-1] connect() @ driver.py:492 - trying: localhost:5672
2010-08-02 13:19:32,071 [WARNING][Thread-1] close_engine() @ driver.py:432 - recoverable error[attempt 137]: [Errno 111] Connection refused
2010-08-02 13:19:32,072 [WARNING][Thread-1] close_engine() @ driver.py:434 - sleeping 120 seconds
2010-08-02 13:19:32,387 [INFO][Actions] __call__() @ action.py:82 - perform "TestAction"
2010-08-02 13:19:32,387 [INFO][Actions] perform() @ actions.py:36 - Hello



2010-08-02 13:20:59,697 [INFO][MainThread] setUp() @ connection.py:135 - Connection Established for cli: Host: localhost, Port: 8811, handler: 
2010-08-02 13:20:59,697 [INFO][MainThread] setUp() @ connection.py:137 - Using cert_file: None and key_file: None
2010-08-02 13:21:02,298 [ERROR][MainThread] _delete() @ core_repo.py:206 - Error: 500: "Traceback (most recent call last):\n  File \"/usr/lib/python2.6/site-packages/pulp-0.0.6-py2.6.egg/pulp/webservices/controllers/base.py\", line 50, in report_error\n    return method(self, *args, **kwargs)\n  File \"/usr/lib/python2.6/site-packages/pulp-0.0.6-py2.6.egg/pulp/webservices/role_check.py\", line 97, in check_roles\n    ret=f(instance, *fargs, **kw)\n  File \"/usr/lib/python2.6/site-packages/pulp-0.0.6-py2.6.egg/pulp/webservices/controllers/repositories.py\", line 142, in DELETE\n    api.delete(id=id)\n  File \"/usr/lib/python2.6/site-packages/pulp-0.0.6-py2.6.egg/pulp/auditing.py\", line 150, in _audit\n    result = method(*args, **kwargs)\n  File \"/usr/lib/python2.6/site-packages/pulp-0.0.6-py2.6.egg/pulp/api/repo.py\", line 121, in delete\n    self.objectdb.remove(repo, safe=True)\n  File \"/usr/lib64/python2.6/site-packages/pymongo/collection.py\", line 379, in remove\n    message.delete(self.__full_name, spec, safe), safe)\n  File \"/usr/lib64/python2.6/site-packages/pymongo/message.py\", line 139, in delete\n    data += bson.BSON.from_dict(spec)\n  File \"/usr/lib64/python2.6/site-packages/pymongo/bson.py\", line 569, in from_dict\n    return cls(_dict_to_bson(dct, check_keys))\nInvalidDocument: document too large - BSON documents are limited to 4 MB\n"
2010-08-02 13:21:32,172 [WARNING][Thread-1] connect() @ driver.py:492 - trying: localhost:5672
2010-08-02 13:21:32,175 [WARNING][Thread-1] close_engine() @ driver.py:432 - recoverable error[attempt 138]: [Errno 111] Connection refused
2010-08-02 13:21:32,175 [WARNING][Thread-1] close_engine() @ driver.py:434 - sleeping 120 seconds
Comment 1 Pradeep Kilambi 2010-08-17 14:41:41 UTC 
this should be fixed. Please retest.
Comment 2 Pradeep Kilambi 2010-08-17 14:43:36 UTC 
*** Bug 621276 has been marked as a duplicate of this bug. ***
Comment 3 Shveta 2010-08-20 09:33:53 UTC 
[root@fedora12g2 ~]# pulp-admin -u admin -p admin repo delete --id=i386
 Deleted operation failed on Repo [ i386 ] 

[root@fedora12g2 grinder]# tail -f /var/log/pulp/pulptools.log 
2010-08-20 15:01:31,705 [WARNING][Thread-1] close_engine() @ driver.py:432 - recoverable error[attempt 139]: [Errno 111] Connection refused



2010-08-20 15:04:28,585 [INFO][MainThread] setUp() @ connection.py:135 - Connection Established for cli: Host: localhost, Port: 8811, handler: 
2010-08-20 15:04:28,687 [INFO][MainThread] setUp() @ connection.py:137 - Using cert_file: None and key_file: None
2010-08-20 15:04:38,338 [ERROR][MainThread] _delete() @ core_repo.py:208 - Error: 500: "Traceback (most recent call last):\n  File \"/usr/lib/python2.6/site-packages/pulp/webservices/controllers/base.py\", line 51, in report_error\n    return method(self, *args, **kwargs)\n  File \"/usr/lib/python2.6/site-packages/pulp/webservices/role_check.py\", line 102, in check_roles\n    ret=f(instance, *fargs, **kw)\n  File \"/usr/lib/python2.6/site-packages/pulp/webservices/controllers/repositories.py\", line 142, in DELETE\n    api.delete(id=id)\n  File \"/usr/lib/python2.6/site-packages/pulp/auditing.py\", line 186, in _audit\n    result = method(*args, **kwargs)\n  File \"/usr/lib/python2.6/site-packages/pulp/api/repo.py\", line 124, in delete\n    repo = self._get_existing_repo(id)\n  File \"/usr/lib/python2.6/site-packages/pulp/api/repo.py\", line 84, in _get_existing_repo\n    raise PulpException(\"No Repo with id: %s found\" % id)\nPulpException: u'No Repo with id: i386 found'\n"
Comment 5 Preethi Thomas 2010-09-10 17:56:29 UTC 
[root@preethi ~]# rpm -qa |grep grinder
grinder-0.0.60-1.fc13.noarch
[root@preethi ~]# rpm -qa |grep pulp
pulp-client-0.0.60-1.fc13.noarch
pulp-0.0.60-1.fc13.noarch
pulp-common-0.0.60-1.fc13.noarch

[root@preethi ~]#  pulp-admin -uadmin -padmin repo sync --id=f13
Task created with ID:: d9078045-bd05-11df-9b4b-002564a85a58
Sync Status:: running
Sync Status:: running
Sync Status:: running
Sync Status:: running
Sync Status:: running
Sync Status:: running
Sync Status:: running
Sync Status:: running
Sync Status:: running
Sync Status:: running
Sync Status:: running
Sync Status:: running
Sync Status:: running
Sync Status:: finished
 Sync Successful. Repo [ f13 ] now has a total of [ 3102 ] packages
You have new mail in /var/spool/mail/root
Comment 6 Preethi Thomas 2010-10-20 16:11:18 UTC 
Closed after review.