Bug 2034317
| Summary: | hammer repository upload-content with large file gives "Too many open files" error | |||
|---|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Ryan Verdile <rverdile> | |
| Component: | Pulp | Assignee: | satellite6-bugs <satellite6-bugs> | |
| Status: | CLOSED ERRATA | QA Contact: | Vladimír Sedmík <vsedmik> | |
| Severity: | medium | Docs Contact: | ||
| Priority: | medium | |||
| Version: | 6.10.0 | CC: | ahumbe, bbuckingham, dkliban, ggainey, osousa, rchan, sadas, saydas, vsedmik | |
| Target Milestone: | 6.11.0 | Keywords: | Triaged | |
| Target Release: | Unused | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | python-pulpcore-3.16.7-1 | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 2059374 (view as bug list) | Environment: | ||
| Last Closed: | 2022-07-05 14:31:12 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
|
Description
Ryan Verdile
2021-12-20 16:26:10 UTC
Steps to Reproduce: 1. Install Satellite 6.10+ 2. Create three files of 1 ,2 and 3 GB using fallocate command. # fallocate -l 1G onegbfile # fallocate -l 2G twogbfile # fallocate -l 3G threegbfile 3. Create a product called "test" in satellite and a file type repo called "test_repo" inside of that product. 4. Try uploading the files using any API or hammer or foreman ansible modules. # hammer repository upload-content --content-type file --path ./onegbfile --id 7 --organization-id 1 # hammer> repository upload-content --content-type file --path ./twogbfile --id 7 --organization-id 1 # hammer repository upload-content --content-type file --path ./threegbfile --id 7 --organization-id 1 Actual results: # du -sh *gbfile 1.0G onegbfile 3.0G threegbfile 2.0G twogbfile # hammer repository upload-content --content-type file --path ./onegbfile --id 7 --organization-id 1 Successfully uploaded file 'onegbfile' # hammer> repository upload-content --content-type file --path ./twogbfile --id 7 --organization-id 1 Successfully uploaded file 'twogbfile' # hammer repository upload-content --content-type file --path ./threegbfile --id 7 --organization-id 1 Could not upload the content: Task bbd22637-59d5-47c7-8bc0-af8d9604e927: Katello::Errors::Pulp3Error: [Errno 24] Too many open files: '/var/lib/pulp/media/upload/f73231bc-bcba-43aa-85ea-644d407d398c' Traceback : Jan 27 17:50:00 satellite610 pulpcore-worker-1: pulp [0f97dc19-aed5-44e7-8e1f-806b0c2dfdca]: pulpcore.tasking.pulpcore_worker:INFO: Task eaa53a92-7b7b-48fe-bc2f-cb44c6d45197 failed ([Errno 24] Too many open files: '/var/lib/pulp/media/upload/f73231bc-bcba-43aa-85ea-644d407d398c') Jan 27 17:50:00 satellite610 pulpcore-worker-1: pulp [0f97dc19-aed5-44e7-8e1f-806b0c2dfdca]: pulpcore.tasking.pulpcore_worker:INFO: File "/usr/lib/python3.6/site-packages/pulpcore/tasking/pulpcore_worker.py", line 317, in _perform_task Jan 27 17:50:00 satellite610 pulpcore-worker-1: result = func(*args, **kwargs) Jan 27 17:50:00 satellite610 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/pulpcore/app/tasks/upload.py", line 31, in commit Jan 27 17:50:00 satellite610 pulpcore-worker-1: temp_file.write(chunk.file.read()) Jan 27 17:50:00 satellite610 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/core/files/utils.py", line 42, in <lambda> Jan 27 17:50:00 satellite610 pulpcore-worker-1: read = property(lambda self: self.file.read) Jan 27 17:50:00 satellite610 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/models/fields/files.py", line 44, in _get_file Jan 27 17:50:00 satellite610 pulpcore-worker-1: self._file = self.storage.open(self.name, 'rb') Jan 27 17:50:00 satellite610 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/core/files/storage.py", line 38, in open Jan 27 17:50:00 satellite610 pulpcore-worker-1: return self._open(name, mode) Jan 27 17:50:00 satellite610 pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/core/files/storage.py", line 231, in _open Jan 27 17:50:00 satellite610 pulpcore-worker-1: return File(open(self.path(name), mode)) Expected results: Satellite\Pulp should allow uploading the 3 GB file without any issues. Additional info: The user who actually had reported the issue has tried to upload a 2.74 GB rpm inside a custom yum repo and that fails with same issue in Satellite 6.10 and this was possible for him in Pulp2 i.e. Sat 6.9. Verified in 6.11.0 snap 17 (tfm-pulpcore-python3-pulpcore-3.16.7-1.el7pc.noarch and python38-pulpcore-3.16.7-1.el8pc.noarch) - successfully uploaded files larger than 3GB via hammer and fetched them from the Satellite's repo. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: Satellite 6.11 Release), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:5498 |