Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Description of problem:
The Pulp 2 to Pulp 3 migration stucks and raises 'pymongo.errors.DocumentTooLarge' error in /var/log/messages. The strange thing is, it only happened once. The error is gone after re-running the 'satellite-maintain content prepare'. I also tried migration-reset and run migration again but unable to reproduce this issue the 2nd time. I think this might happen to some users having large number of errata so I created this bugzilla for further investigation if needed.
# satellite-maintain content prepare
...
Prepare content for Pulp 3:
Checking for valid Katello configuraton.
Starting task.
2021-08-31 15:01:37 +1000: Pre-migrating Pulp 2 erratum content (detail info) 601855/602875
pulpcore-worker-4: RuntimeWarning)
pulpcore-worker-4: pulp: rq.worker:ERROR: Traceback (most recent call last):
pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job
pulpcore-worker-4: rv = job.perform()
pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform
pulpcore-worker-4: self._result = self._execute()
pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute
pulpcore-worker-4: return self.func(*self.args, **self.kwargs)
pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py", line 77, in migrate_from_pulp2
pulpcore-worker-4: pre_migrate_all_content(plan)
pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/pre_migration.py", line 70, in pre_migrate_all_content
pulpcore-worker-4: pre_migrate_content_type(content_model, mutable_type, lazy_type, premigrate_hook)
pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/pre_migration.py", line 300, in pre_migrate_content_type
pulpcore-worker-4: record.id: record for record in mongo_content_qs.only(*mongo_fields).no_cache()
pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/pre_migration.py", line 299, in <dictcomp>
pulpcore-worker-4: pulp2_content_by_id = {
pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/mongoengine/queryset/base.py", line 1590, in __next__
pulpcore-worker-4: raw_doc = next(self._cursor)
pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/cursor.py", line 1207, in next
pulpcore-worker-4: if len(self.__data) or self._refresh():
pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/cursor.py", line 1124, in _refresh
pulpcore-worker-4: self.__send_message(q)
pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/cursor.py", line 1001, in __send_message
pulpcore-worker-4: address=self.__address)
pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/mongo_client.py", line 1372, in _run_operation_with_response
pulpcore-worker-4: exhaust=exhaust)
pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/mongo_client.py", line 1471, in _retryable_read
pulpcore-worker-4: return func(session, server, sock_info, slave_ok)
pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/mongo_client.py", line 1366, in _cmd
pulpcore-worker-4: unpack_res)
pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/server.py", line 116, in run_operation_with_response
pulpcore-worker-4: sock_info.send_message(data, max_doc_size)
pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/pool.py", line 711, in send_message
pulpcore-worker-4: (max_doc_size, self.max_bson_size))
pulpcore-worker-4: pymongo.errors.DocumentTooLarge: BSON document too large (32444638 bytes) - the connected server supports BSON document sizes up to 16777216 bytes.
Note: Since this issue is difficult to reproduce, my testing is limited. I was only able to get around 8.5k erratas in my testing.
Steps to retest
1. Sync several repos with a lot of erratas
2. Run the 'satellite-maintain content prepare'
3. Ensure that migration is successful
Expected result:
Migration should be successful
Actual result:
Migration is successful
Verified on 6.9.8_01
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory (Satellite 6.9.8 Async Bug Fix Update), and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHBA-2022:0320
Description of problem: The Pulp 2 to Pulp 3 migration stucks and raises 'pymongo.errors.DocumentTooLarge' error in /var/log/messages. The strange thing is, it only happened once. The error is gone after re-running the 'satellite-maintain content prepare'. I also tried migration-reset and run migration again but unable to reproduce this issue the 2nd time. I think this might happen to some users having large number of errata so I created this bugzilla for further investigation if needed. # satellite-maintain content prepare ... Prepare content for Pulp 3: Checking for valid Katello configuraton. Starting task. 2021-08-31 15:01:37 +1000: Pre-migrating Pulp 2 erratum content (detail info) 601855/602875 pulpcore-worker-4: RuntimeWarning) pulpcore-worker-4: pulp: rq.worker:ERROR: Traceback (most recent call last): pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job pulpcore-worker-4: rv = job.perform() pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform pulpcore-worker-4: self._result = self._execute() pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute pulpcore-worker-4: return self.func(*self.args, **self.kwargs) pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py", line 77, in migrate_from_pulp2 pulpcore-worker-4: pre_migrate_all_content(plan) pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/pre_migration.py", line 70, in pre_migrate_all_content pulpcore-worker-4: pre_migrate_content_type(content_model, mutable_type, lazy_type, premigrate_hook) pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/pre_migration.py", line 300, in pre_migrate_content_type pulpcore-worker-4: record.id: record for record in mongo_content_qs.only(*mongo_fields).no_cache() pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/pre_migration.py", line 299, in <dictcomp> pulpcore-worker-4: pulp2_content_by_id = { pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/mongoengine/queryset/base.py", line 1590, in __next__ pulpcore-worker-4: raw_doc = next(self._cursor) pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/cursor.py", line 1207, in next pulpcore-worker-4: if len(self.__data) or self._refresh(): pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/cursor.py", line 1124, in _refresh pulpcore-worker-4: self.__send_message(q) pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/cursor.py", line 1001, in __send_message pulpcore-worker-4: address=self.__address) pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/mongo_client.py", line 1372, in _run_operation_with_response pulpcore-worker-4: exhaust=exhaust) pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/mongo_client.py", line 1471, in _retryable_read pulpcore-worker-4: return func(session, server, sock_info, slave_ok) pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/mongo_client.py", line 1366, in _cmd pulpcore-worker-4: unpack_res) pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/server.py", line 116, in run_operation_with_response pulpcore-worker-4: sock_info.send_message(data, max_doc_size) pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/pool.py", line 711, in send_message pulpcore-worker-4: (max_doc_size, self.max_bson_size)) pulpcore-worker-4: pymongo.errors.DocumentTooLarge: BSON document too large (32444638 bytes) - the connected server supports BSON document sizes up to 16777216 bytes.