Description of problem: Pulp3 is unable to handle the sync of repositories with big metadata and RQ invokes OOM at some point during the same which silently kills the Sync Task with the error "Katello::Errors::Pulp3Error : None" Version-Release number of selected component (if applicable): Satellite 6.9.0 Latest Snap [ 19.1 ] with pulp3 enabled Katello 4.0 + Pulp3 How reproducible: Always Steps to Reproduce: 1. Install the latest snap of Satellite 6.9.0 with 2 vCPU and ~12 or ~15 GB of ram 2. Enable pulp3 on the same setup 3. Enable the "Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server" repository and attempt to sync the same. Actual results: While satellite is idle and only that sync is happening, after some time the sync task will fail with the error "None" or to be specific "Katello::Errors::Pulp3Error : None" as RQ invokes OOM. The OOM logs will be attached to the Bugzilla. Part of what RQ logs in Syslog, ~~ kernel: Out of memory: Kill process 7509 (rq) score 374 or sacrifice child kernel: Killed process 7509 (rq), UID 1000, total-vm:6863868kB, anon-rss:5203516kB, file-rss:212kB, shmem-rss:0kB pulpcore-worker-1: pulp: rq.worker:WARNING: Moving job to FailedJobRegistry (work-horse terminated unexpectedly; waitpid returned 9) ~~ Expected results: The repository should get synced even if it takes more than the expected amount of time. Additional info: The same scenario with pulp 2, will take around 8 hours to sync the repo but It will be finished and we will never get to see any OOM but with pulp3 that is not the case. I will add some more info in the private comment of the Bugzilla.
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.
The Pulp upstream bug priority is at High. Updating the external tracker on this bug.
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.
All upstream Pulp bugs are at MODIFIED+. Moving this bug to POST.
The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.
Requesting needsinfo from upstream developer ttereshc because the 'FailedQA' flag is set.
One last question. Are these just syncs or syncs + publishes?
*** Bug 1972769 has been marked as a duplicate of this bug. ***
Resolved in pulp_rpm 3.13
Verified on 6.10.0 snap 11.0. I was able to sync several repos simultaneously, with no errors: Red Hat Enterprise Linux 7 Server - Optional RPMs x86_64 7Server Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server Red Hat Enterprise Linux 7 Server - Extras RPMs x86_64 Red Hat Satellite Tools 6.9 for RHEL 7 Server RPMs x86_64 Red Hat Enterprise Linux 8 for x86_64 - AppStream RPMs 8 Red Hat Enterprise Linux 8 for x86_64 - BaseOS RPMs 8 Red Hat Satellite Capsule 6.9 for RHEL 7 Server RPMs x86_64
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: Satellite 6.10 Release), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:4702