Bug 2003888
| Summary: | The Pulp 2 to Pulp 3 migration stucks and raises 'pymongo.errors.DocumentTooLarge' error in /var/log/messages | ||
|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Hao Chang Yu <hyu> |
| Component: | Pulp | Assignee: | satellite6-bugs <satellite6-bugs> |
| Status: | CLOSED ERRATA | QA Contact: | Lai <ltran> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 6.9.0 | CC: | ahumbe, ajambhul, alsouza, arsingh, ggainey, gpadholi, ipanova, jhanley, ldelouw, momran, nikhjain, pmendezh, rchan, saydas, ttereshc, vcojot, zhunting |
| Target Milestone: | 6.9.8 | Keywords: | Triaged, Upgrades |
| Target Release: | Unused | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-01-27 17:33:09 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Without steps to reproduce or more information about the repos and the content that was migrating I am not sure how we can proceed. The Pulp upstream bug status is at POST. Updating the external tracker on this bug. The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug. The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug. The Pulp upstream bug status is at NEW. Updating the external tracker on this bug. The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug. The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug. All upstream Pulp bugs are at MODIFIED+. Moving this bug to POST. The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug. *** Bug 2032684 has been marked as a duplicate of this bug. *** Note: Since this issue is difficult to reproduce, my testing is limited. I was only able to get around 8.5k erratas in my testing. Steps to retest 1. Sync several repos with a lot of erratas 2. Run the 'satellite-maintain content prepare' 3. Ensure that migration is successful Expected result: Migration should be successful Actual result: Migration is successful Verified on 6.9.8_01 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Satellite 6.9.8 Async Bug Fix Update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:0320 |
Description of problem: The Pulp 2 to Pulp 3 migration stucks and raises 'pymongo.errors.DocumentTooLarge' error in /var/log/messages. The strange thing is, it only happened once. The error is gone after re-running the 'satellite-maintain content prepare'. I also tried migration-reset and run migration again but unable to reproduce this issue the 2nd time. I think this might happen to some users having large number of errata so I created this bugzilla for further investigation if needed. # satellite-maintain content prepare ... Prepare content for Pulp 3: Checking for valid Katello configuraton. Starting task. 2021-08-31 15:01:37 +1000: Pre-migrating Pulp 2 erratum content (detail info) 601855/602875 pulpcore-worker-4: RuntimeWarning) pulpcore-worker-4: pulp: rq.worker:ERROR: Traceback (most recent call last): pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 936, in perform_job pulpcore-worker-4: rv = job.perform() pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/job.py", line 684, in perform pulpcore-worker-4: self._result = self._execute() pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/job.py", line 690, in _execute pulpcore-worker-4: return self.func(*self.args, **self.kwargs) pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/tasks/migrate.py", line 77, in migrate_from_pulp2 pulpcore-worker-4: pre_migrate_all_content(plan) pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/pre_migration.py", line 70, in pre_migrate_all_content pulpcore-worker-4: pre_migrate_content_type(content_model, mutable_type, lazy_type, premigrate_hook) pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/pre_migration.py", line 300, in pre_migrate_content_type pulpcore-worker-4: record.id: record for record in mongo_content_qs.only(*mongo_fields).no_cache() pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/pulp_2to3_migration/app/pre_migration.py", line 299, in <dictcomp> pulpcore-worker-4: pulp2_content_by_id = { pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/mongoengine/queryset/base.py", line 1590, in __next__ pulpcore-worker-4: raw_doc = next(self._cursor) pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/cursor.py", line 1207, in next pulpcore-worker-4: if len(self.__data) or self._refresh(): pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/cursor.py", line 1124, in _refresh pulpcore-worker-4: self.__send_message(q) pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/cursor.py", line 1001, in __send_message pulpcore-worker-4: address=self.__address) pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/mongo_client.py", line 1372, in _run_operation_with_response pulpcore-worker-4: exhaust=exhaust) pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/mongo_client.py", line 1471, in _retryable_read pulpcore-worker-4: return func(session, server, sock_info, slave_ok) pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/mongo_client.py", line 1366, in _cmd pulpcore-worker-4: unpack_res) pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/server.py", line 116, in run_operation_with_response pulpcore-worker-4: sock_info.send_message(data, max_doc_size) pulpcore-worker-4: File "/usr/lib64/python3.6/site-packages/pymongo/pool.py", line 711, in send_message pulpcore-worker-4: (max_doc_size, self.max_bson_size)) pulpcore-worker-4: pymongo.errors.DocumentTooLarge: BSON document too large (32444638 bytes) - the connected server supports BSON document sizes up to 16777216 bytes.