Description of problem: Uploading ~3gb content with hammer throws the error: "Katello::Errors::Pulp3Error: [Errno 24] Too many open files:" Version-Release number of selected component (if applicable): How reproducible: 100% Steps to Reproduce: 1. Download a large file e.g. `wget https://releases.ubuntu.com/20.04/ubuntu-20.04.3-desktop-amd64.iso` (2.9gb) 2. hammer repository upload-content --id <repo-id> --content-type file --path ubuntu-20.04.3-desktop-amd64.iso Actual results: Could not upload the content: Task bab20504-bedc-49ba-a138-e3b17cc44375: Katello::Errors::Pulp3Error: [Errno 24] Too many open files: '/var/lib/pulp/media/upload/88092f38-9876-45ec-a713-da3713231f11' Expected results: File uploads successfully Additional info:
Steps to Reproduce: 1. Install Satellite 6.10+ 2. Create three files of 1 ,2 and 3 GB using fallocate command. # fallocate -l 1G onegbfile # fallocate -l 2G twogbfile # fallocate -l 3G threegbfile 3. Create a product called "test" in satellite and a file type repo called "test_repo" inside of that product. 4. Try uploading the files using any API or hammer or foreman ansible modules. # hammer repository upload-content --content-type file --path ./onegbfile --id 7 --organization-id 1 # hammer> repository upload-content --content-type file --path ./twogbfile --id 7 --organization-id 1 # hammer repository upload-content --content-type file --path ./threegbfile --id 7 --organization-id 1 Actual results: # du -sh *gbfile 1.0G onegbfile 3.0G threegbfile 2.0G twogbfile # hammer repository upload-content --content-type file --path ./onegbfile --id 7 --organization-id 1 Successfully uploaded file 'onegbfile' # hammer> repository upload-content --content-type file --path ./twogbfile --id 7 --organization-id 1 Successfully uploaded file 'twogbfile' # hammer repository upload-content --content-type file --path ./threegbfile --id 7 --organization-id 1 Could not upload the content: Task bbd22637-59d5-47c7-8bc0-af8d9604e927: Katello::Errors::Pulp3Error: [Errno 24] Too many open files: '/var/lib/pulp/media/upload/f73231bc-bcba-43aa-85ea-644d407d398c' Traceback : Jan 27 17:50:00 satellite610 pulpcore-worker-1: pulp [0f97dc19-aed5-44e7-8e1f-806b0c2dfdca]: pulpcore.tasking.pulpcore_worker:INFO: Task eaa53a92-7b7b-48fe-bc2f-cb44c6d45197 failed ([Errno 24] Too many open files: '/var/lib/pulp/media/upload/f73231bc-bcba-43aa-85ea-644d407d398c') Jan 27 17:50:00 satellite610 pulpcore-worker-1: pulp [0f97dc19-aed5-44e7-8e1f-806b0c2dfdca]: pulpcore.tasking.pulpcore_worker:INFO: File "/usr/lib/python3.6/site-packages/pulpcore/tasking/pulpcore_worker.py", line 317, in _perform_task Jan 27 17:50:00 satellite610 pulpcore-worker-1: result = func(*args, **kwargs) Jan 27 17:50:00 satellite610 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulpcore/app/tasks/upload.py", line 31, in commit Jan 27 17:50:00 satellite610 pulpcore-worker-1: temp_file.write(chunk.file.read()) Jan 27 17:50:00 satellite610 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/core/files/utils.py", line 42, in <lambda> Jan 27 17:50:00 satellite610 pulpcore-worker-1: read = property(lambda self: self.file.read) Jan 27 17:50:00 satellite610 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/fields/files.py", line 44, in _get_file Jan 27 17:50:00 satellite610 pulpcore-worker-1: self._file = self.storage.open(self.name, 'rb') Jan 27 17:50:00 satellite610 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/core/files/storage.py", line 38, in open Jan 27 17:50:00 satellite610 pulpcore-worker-1: return self._open(name, mode) Jan 27 17:50:00 satellite610 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/core/files/storage.py", line 231, in _open Jan 27 17:50:00 satellite610 pulpcore-worker-1: return File(open(self.path(name), mode)) Expected results: Satellite\Pulp should allow uploading the 3 GB file without any issues. Additional info: The user who actually had reported the issue has tried to upload a 2.74 GB rpm inside a custom yum repo and that fails with same issue in Satellite 6.10 and this was possible for him in Pulp2 i.e. Sat 6.9.
Verified in 6.11.0 snap 17 (tfm-pulpcore-python3-pulpcore-3.16.7-1.el7pc.noarch and python38-pulpcore-3.16.7-1.el8pc.noarch) - successfully uploaded files larger than 3GB via hammer and fetched them from the Satellite's repo.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: Satellite 6.11 Release), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:5498