Bug 2240519

Summary: Pulp repair task can fail immediately with 404 not found error
Product: Red Hat Satellite Reporter: Hao Chang Yu <hyu>
Component: PulpAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED DUPLICATE QA Contact: Satellite QE Team <sat-qe-bz-list>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.11.5CC: dalley, rlavi
Target Milestone: UnspecifiedKeywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-10-13 16:40:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Hao Chang Yu 2023-09-25 03:52:48 UTC
Description of problem:
The repair task will fail immediately with 404 Not Found error if a content has been deleted in the upstream repository, 
---------------------------------
    "error": {
        "traceback": "  File "/usr/lib/python3.8/site-packages/pulpcore/tasking/pulpcore_worker.py", line 380, in _perform_task
             result = func(*args, **kwargs)
           File "/usr/lib/python3.8/site-packages/pulpcore/app/tasks/repository.py", line 194, in repair_all_artifacts
             loop.run_until_complete(_repair_artifacts_for_content(verify_checksums=verify_checksums))
           File "/usr/lib64/python3.8/asyncio/base_events.py", line 616, in run_until_complete
             return future.result()
           File "/usr/lib/python3.8/site-packages/pulpcore/app/tasks/repository.py", line 150, in _repair_artifacts_for_content
             await asyncio.gather(*done)  # Clean up tasks
           File "/usr/lib/python3.8/site-packages/pulpcore/app/tasks/repository.py", line 78, in _repair_ca
             dl_result = await downloader.run()
           File "/usr/lib/python3.8/site-packages/pulpcore/download/http.py", line 273, in run
             return await download_wrapper()
           File "/usr/lib/python3.8/site-packages/backoff/_async.py", line 133, in retry
             ret = await target(*args, **kwargs)
           File "/usr/lib/python3.8/site-packages/pulpcore/download/http.py", line 258, in download_wrapper
             return await self._run(extra_data=extra_data)
           File "/usr/lib/python3.8/site-packages/pulp_rpm/app/downloaders.py", line 117, in _run
             self.raise_for_status(response)
           File "/usr/lib/python3.8/site-packages/pulp_rpm/app/downloaders.py", line 102, in raise_for_status
             response.raise_for_status()
           File "/usr/lib64/python3.8/site-packages/aiohttp/client_reqrep.py", line 1004, in raise_for_status
             raise ClientResponseError(
         ",
        "description": "404, message='Not Found', url=URL('https://dl.fedoraproject.org/pub/epel/9/Everything/x86_64/Packages/c/chromium-common-114.0.5735.106-1.el9.x86_64.rpm')"
---------------------------------


It will be good and convenient if the repair task can tolerate the 404 not found error and continue to download other missing contents. Otherwise, user will have to repair each repository one by one.

Comment 1 Daniel Alley 2023-10-13 16:40:40 UTC

*** This bug has been marked as a duplicate of bug 2173692 ***

Comment 2 Daniel Alley 2023-10-13 16:42:11 UTC
I noticed that this is filed against 6.11.  The fix for the BZ this is a duplicate of isn't backported back that far, so if you really need a fix on 6.11 and cannot upgrade to a release that is getting that fix, please request it on that BZ.