Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2173671 - [Pulp-3] Repair API cannot fix\re-download the docker repo content completely
Summary: [Pulp-3] Repair API cannot fix\re-download the docker repo content completely
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Pulp
Version: 6.13.0
Hardware: All
OS: All
medium
high
Target Milestone: 6.14.0
Assignee: satellite6-bugs
QA Contact: visawant
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-02-27 15:35 UTC by Sayan Das
Modified: 2023-11-08 14:18 UTC (History)
7 users (show)

Fixed In Version: pulp-container-2.14.6-1, pulp-container-2.15.1-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-11-08 14:18:33 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github pulp pulpcore issues 3606 0 None closed Pulp repair does not work with container manifests 2023-06-16 14:18:02 UTC
Red Hat Issue Tracker SAT-16328 0 None None None 2023-03-06 14:25:53 UTC
Red Hat Product Errata RHSA-2023:6818 0 None None None 2023-11-08 14:18:48 UTC

Description Sayan Das 2023-02-27 15:35:29 UTC 
Description of problem:

Repair API of Pulp3 ( Verify Content Checksum of Katello ) cannot get back\fix docker manifests 


Version-Release number of selected component (if applicable):

satellite-6.13.0-6.el8sat.noarch
katello-4.7.0-1.el8sat.noarch
python39-pulpcore-3.21.5-1.1.el8pc.noarch
python39-pulp-rpm-3.18.11-1.el8pc.noarch


How reproducible:
Always

Steps to Reproduce:

1. Install a Satellite 6.13 and import a manifest with some valid subs
2. Use the repo_discovery feature to discover the "rhel8/toolbox" repo and create it inside the Containers product
3. Go to Satellite UI Content --> Products --> Containers --> Repositories --> rhel8/toolbox and add "8.7-5" in the "Include Tags" fields and sync the repo. ( It will only sync that tag now )
4. Delete the artifacts 
   # rm -rf /var/lib/pulp/media/artifact/*
   
5. Go to Satellite UI Content --> Products --> Containers --> Repositories --> rhel8/toolbox --> Select Action dropdown --> Sync the repo.
  
6. Go to Satellite UI Content --> Products --> Containers --> Repositories --> rhel8/toolbox --> Select Action dropdown --> Verify Content Checksum

   Or else just invoke the Repair API 
   
   # curl --cert /etc/foreman/client_cert.pem --key /etc/foreman/client_key.pem -H "Content-Type: application/json" -X POST https://`hostname -f`/pulp/api/v3/repair/ | json_reformat


Actual results:

Step 3: Sync status from Sat UI:

Added Docker tags: 1
Total steps: 32/32
--------------------------------
Associating Content: 18/18
Downloading Artifacts: 12/12
Downloading tag list: 1/1
Processing Tags: 1/1
Un-Associating Content: 0/0


# du -sh /var/lib/pulp/media/artifact/
556M	/var/lib/pulp/media/artifact/

# file /var/lib/pulp/media/artifact/*/*
/var/lib/pulp/media/artifact/01/65232f07e945f1c1831eebb8c42e5695f80d47d7c3d787af20752c89333276: ASCII text, with very long lines, with no line terminators
/var/lib/pulp/media/artifact/48/fd57131d1a3ccda35d72ca802a44ced559941d9fe5ee160dbe3df0414e7cb3: ASCII text
/var/lib/pulp/media/artifact/5d/88a016637781e6d3eeb22c15c788ce1cc572e2dbb7bad4935d37a775c252d9: gzip compressed data, original size 169271808
/var/lib/pulp/media/artifact/62/9545c824752b6d75f7f22a73b6c7fcba9413b6017f2f67f58494a958bcd4ce: ASCII text, with very long lines, with no line terminators
/var/lib/pulp/media/artifact/64/554374912209545e580ebabc0c025363b6982d1fbffb00c50edfeea18e79b9: gzip compressed data, original size 212109824
/var/lib/pulp/media/artifact/6d/2cf55f7908bf17bd221297c8cff9750fad3187c3eb102d1c3f72a208efbf0b: gzip compressed data, original size 235748864
/var/lib/pulp/media/artifact/70/2f3a622329ec2d54c1bf3c13fa094c422d48abaad700a2e055c3b77bea15c0: UTF-8 Unicode text, with very long lines, with no line terminators
/var/lib/pulp/media/artifact/7f/060f344d579611663e84ea94d89d9372d7431e3e56e11a19e704207d5576e9: ASCII text, with very long lines, with no line terminators
/var/lib/pulp/media/artifact/9a/394c72313c9bcac63dc8980018582d90ece7ae65d03940e6d7e9b8d4cb884d: gzip compressed data, original size 264029696
/var/lib/pulp/media/artifact/a5/d805d6872a1693a518e623b95182f115915c37671f70c1b13eb136e930bfe4: gzip compressed data, original size 204123648
/var/lib/pulp/media/artifact/ac/88d64d7b415280e8c1221725a8455917b99711b754a5981d3460fa9fdf40f9: UTF-8 Unicode text, with very long lines, with no line terminators
/var/lib/pulp/media/artifact/b1/45dc70f6c036069d549762f39055162cac12b414e9e502fd0d4b3331a0f5eb: gzip compressed data, original size 171049472
/var/lib/pulp/media/artifact/b9/2727ef7443d73bb77323e5a969bf02187286fccf53930e83dc82be60a36529: gzip compressed data, original size 214139392
/var/lib/pulp/media/artifact/c6/25ce0354e96c43d4bfca9cb0a3ed8373b0707980f152582f731e61808df9bd: ASCII text, with very long lines, with no line terminators
/var/lib/pulp/media/artifact/d0/898311a8acc605593edbc1f37171f42aab926fffd820a0f0460e062a4f6717: UTF-8 Unicode text, with very long lines, with no line terminators
/var/lib/pulp/media/artifact/d4/d59e4eb38ff5eda9a9f6255575abc64501adb78153f5788bf580b1034d4f8d: gzip compressed data, original size 186984960
/var/lib/pulp/media/artifact/d5/2583c79ff512b040d97ca0ce70b61b9c496e674b789f8857efbcf93727e4b0: UTF-8 Unicode text, with very long lines, with no line terminators



Step 4: 


# rm -rf /var/lib/pulp/media/artifact/*

# du -sh /var/lib/pulp/media/artifact/
0	/var/lib/pulp/media/artifact/


Step 5:

Sync fails but able to get some data in pulp:

# file /var/lib/pulp/media/artifact/*/*
/var/lib/pulp/media/artifact/48/fd57131d1a3ccda35d72ca802a44ced559941d9fe5ee160dbe3df0414e7cb3: ASCII text


Step 6:

# curl --cert /etc/foreman/client_cert.pem --key /etc/foreman/client_key.pem -H "Content-Type: application/json" -X POST https://`hostname -f`/pulp/api/v3/repair/ | json_reformat
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    67  100    67    0     0    881      0 --:--:-- --:--:-- --:--:--   881
{
    "task": "/pulp/api/v3/tasks/bff6cd3c-c7cb-43c1-a717-ec68512e10fa/"
}


# curl --cert /etc/foreman/client_cert.pem --key /etc/foreman/client_key.pem -H "Content-Type: application/json" -X GET https://`hostname -f`/pulp/api/v3/tasks/bff6cd3c-c7cb-43c1-a717-ec68512e10fa/ | json_reformat
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2860  100  2860    0     0  46885      0 --:--:-- --:--:-- --:--:-- 46885
{
    "pulp_href": "/pulp/api/v3/tasks/bff6cd3c-c7cb-43c1-a717-ec68512e10fa/",
    "pulp_created": "2023-02-27T15:30:24.465374Z",
    "state": "failed",
    "name": "pulpcore.app.tasks.repository.repair_all_artifacts",
    "logging_cid": "15f07cd0ac6146968f91e6b229189b3f",
    "started_at": "2023-02-27T15:30:24.508613Z",
    "finished_at": "2023-02-27T15:30:26.289194Z",
    "error": {
        "traceback": "  File \"/usr/lib/python3.9/site-packages/pulpcore/tasking/pulpcore_worker.py\", line 452, in _perform_task\n    result = func(*args, **kwargs)\n  File \"/usr/lib/python3.9/site-packages/pulpcore/app/tasks/repository.py\", line 192, in repair_all_artifacts\n    loop.run_until_complete(_repair_artifacts_for_content(verify_checksums=verify_checksums))\n  File \"/usr/lib64/python3.9/asyncio/base_events.py\", line 647, in run_until_complete\n    return future.result()\n  File \"/usr/lib/python3.9/site-packages/pulpcore/app/tasks/repository.py\", line 148, in _repair_artifacts_for_content\n    await asyncio.gather(*done)  # Clean up tasks\n  File \"/usr/lib/python3.9/site-packages/pulpcore/app/tasks/repository.py\", line 77, in _repair_ca\n    dl_result = await downloader.run()\n  File \"/usr/lib/python3.9/site-packages/pulpcore/download/http.py\", line 273, in run\n    return await download_wrapper()\n  File \"/usr/lib/python3.9/site-packages/backoff/_async.py\", line 151, in retry\n    ret = await target(*args, **kwargs)\n  File \"/usr/lib/python3.9/site-packages/pulpcore/download/http.py\", line 258, in download_wrapper\n    return await self._run(extra_data=extra_data)\n  File \"/usr/lib/python3.9/site-packages/pulp_container/app/downloaders.py\", line 81, in _run\n    return await self._run(handle_401=False, extra_data=extra_data)\n  File \"/usr/lib/python3.9/site-packages/pulp_container/app/downloaders.py\", line 66, in _run\n    response.raise_for_status()\n  File \"/usr/lib64/python3.9/site-packages/aiohttp/client_reqrep.py\", line 1004, in raise_for_status\n    raise ClientResponseError(\n",
        "description": "404, message='Not Found', url=URL('https://registry.redhat.io/containers/content/dist/containers/rhel8/multiarch/containers/redhat-rhel8-toolbox/manifests/1/sha256:0165232f07e945f1c1831eebb8c42e5695f80d47d7c3d787af20752c89333276?_auth_=exp=1677515426~hmac=8110d87e5268564dd809419bef770002c546ec040bf0f13e8733d4ae9ded8c6d')"
    },
    "worker": "/pulp/api/v3/workers/854914bf-6e9d-4bd9-8c84-d13557f44e82/",
    "parent_task": null,
    "child_tasks": [

    ],
    "task_group": null,
    "progress_reports": [
        {
            "message": "Repair corrupted units",
            "code": "repair.repaired",
            "state": "failed",
            "total": null,
            "done": 0,
            "suffix": null
        },
        {
            "message": "Identify corrupted units",
            "code": "repair.corrupted",
            "state": "failed",
            "total": null,
            "done": 0,
            "suffix": null
        },
        {
            "message": "Identify missing units",
            "code": "repair.missing",
            "state": "failed",
            "total": null,
            "done": 6,
            "suffix": null
        }
    ],
    "created_resources": [

    ],
    "reserved_resources_record": [

    ]
}




Trace from logs:

Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]: pulp [15f07cd0ac6146968f91e6b229189b3f]: pulp_container.app.downloaders:INFO: Updating bearer token
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]: Giving up download_wrapper(...) after 1 tries (aiohttp.client_exceptions.ClientResponseError: 404, message='Not Found', url=URL('https://registry.redhat.io/containers/content/dist/containers/rhel8/multiarch/containers/redhat-rhel8-toolbox/manifests/1/sha256:0165232f07e945f1c1831eebb8c42e5695f80d47d7c3d787af20752c89333276?_auth_=exp=1677515426~hmac=8110d87e5268564dd809419bef770002c546ec040bf0f13e8733d4ae9ded8c6d'))
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]: pulp [15f07cd0ac6146968f91e6b229189b3f]: backoff:ERROR: Giving up download_wrapper(...) after 1 tries (aiohttp.client_exceptions.ClientResponseError: 404, message='Not Found', url=URL('https://registry.redhat.io/containers/content/dist/containers/rhel8/multiarch/containers/redhat-rhel8-toolbox/manifests/1/sha256:0165232f07e945f1c1831eebb8c42e5695f80d47d7c3d787af20752c89333276?_auth_=exp=1677515426~hmac=8110d87e5268564dd809419bef770002c546ec040bf0f13e8733d4ae9ded8c6d'))
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]: pulp [15f07cd0ac6146968f91e6b229189b3f]: pulpcore.tasking.pulpcore_worker:INFO: Task bff6cd3c-c7cb-43c1-a717-ec68512e10fa failed (404, message='Not Found', url=URL('https://registry.redhat.io/containers/content/dist/containers/rhel8/multiarch/containers/redhat-rhel8-toolbox/manifests/1/sha256:0165232f07e945f1c1831eebb8c42e5695f80d47d7c3d787af20752c89333276?_auth_=exp=1677515426~hmac=8110d87e5268564dd809419bef770002c546ec040bf0f13e8733d4ae9ded8c6d'))
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]: pulp [15f07cd0ac6146968f91e6b229189b3f]: pulpcore.tasking.pulpcore_worker:INFO:   File "/usr/lib/python3.9/site-packages/pulpcore/tasking/pulpcore_worker.py", line 452, in _perform_task
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]:    result = func(*args, **kwargs)
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]:  File "/usr/lib/python3.9/site-packages/pulpcore/app/tasks/repository.py", line 192, in repair_all_artifacts
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]:    loop.run_until_complete(_repair_artifacts_for_content(verify_checksums=verify_checksums))
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]:  File "/usr/lib64/python3.9/asyncio/base_events.py", line 647, in run_until_complete
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]:    return future.result()
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]:  File "/usr/lib/python3.9/site-packages/pulpcore/app/tasks/repository.py", line 148, in _repair_artifacts_for_content
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]:    await asyncio.gather(*done)  # Clean up tasks
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]:  File "/usr/lib/python3.9/site-packages/pulpcore/app/tasks/repository.py", line 77, in _repair_ca
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]:    dl_result = await downloader.run()
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]:  File "/usr/lib/python3.9/site-packages/pulpcore/download/http.py", line 273, in run
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]:    return await download_wrapper()
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]:  File "/usr/lib/python3.9/site-packages/backoff/_async.py", line 151, in retry
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]:    ret = await target(*args, **kwargs)
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]:  File "/usr/lib/python3.9/site-packages/pulpcore/download/http.py", line 258, in download_wrapper
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]:    return await self._run(extra_data=extra_data)
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]:  File "/usr/lib/python3.9/site-packages/pulp_container/app/downloaders.py", line 81, in _run
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]:    return await self._run(handle_401=False, extra_data=extra_data)
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]:  File "/usr/lib/python3.9/site-packages/pulp_container/app/downloaders.py", line 66, in _run
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]:    response.raise_for_status()
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]:  File "/usr/lib64/python3.9/site-packages/aiohttp/client_reqrep.py", line 1004, in raise_for_status
Feb 27 21:00:26 satellite613 pulpcore-worker-1[75377]:    raise ClientResponseError(



Filesystem state:

# file /var/lib/pulp/media/artifact/*/*
/var/lib/pulp/media/artifact/48/fd57131d1a3ccda35d72ca802a44ced559941d9fe5ee160dbe3df0414e7cb3: ASCII text
/var/lib/pulp/media/artifact/5d/88a016637781e6d3eeb22c15c788ce1cc572e2dbb7bad4935d37a775c252d9: gzip compressed data, original size 169271808
/var/lib/pulp/media/artifact/64/554374912209545e580ebabc0c025363b6982d1fbffb00c50edfeea18e79b9: gzip compressed data, original size 212109824
/var/lib/pulp/media/artifact/6d/2cf55f7908bf17bd221297c8cff9750fad3187c3eb102d1c3f72a208efbf0b: gzip compressed data, original size 235748864
/var/lib/pulp/media/artifact/70/2f3a622329ec2d54c1bf3c13fa094c422d48abaad700a2e055c3b77bea15c0: UTF-8 Unicode text, with very long lines, with no line terminators
/var/lib/pulp/media/artifact/a5/d805d6872a1693a518e623b95182f115915c37671f70c1b13eb136e930bfe4: gzip compressed data, original size 204123648
/var/lib/pulp/media/artifact/ac/88d64d7b415280e8c1221725a8455917b99711b754a5981d3460fa9fdf40f9: UTF-8 Unicode text, with very long lines, with no line terminators
/var/lib/pulp/media/artifact/d0/898311a8acc605593edbc1f37171f42aab926fffd820a0f0460e062a4f6717: UTF-8 Unicode text, with very long lines, with no line terminators
/var/lib/pulp/media/artifact/d5/2583c79ff512b040d97ca0ce70b61b9c496e674b789f8857efbcf93727e4b0: UTF-8 Unicode text, with very long lines, with no line terminators


Expected results:

The Repair API should be able to gracefully get back all the syncable data ( even for container\docker type repos )


Additional info:

* Reproducer details shared in the next private comment

* It has been noticed that, if this API fails to fix something, Then it immediately ends the task instead of checking for the other fixable objects\units and trying to fix\redownload them.
Comment 3 Brad Buckingham 2023-03-02 15:32:18 UTC 
Is this a regression from 6.12?  Thanks!
Comment 4 Sayan Das 2023-03-02 15:38:41 UTC 
Not a regression for sure.

My answer remains same as https://bugzilla.redhat.com/show_bug.cgi?id=2173692#c3
Comment 6 visawant 2023-07-06 10:49:12 UTC 
Verified


Satellite & package version:
----------------------------
satellite-installer-6.14.0.5-1.el8sat.noarch
satellite-maintain-0.0.2-1.el8sat.noarch
satellite-6.14.0-2.el8sat.noarch
satellite-cli-6.14.0-2.el8sat.noarch
python39-pulp-container-2.14.6-1.el8pc.noarch
python39-pulp-container-2.14.6-1.el8pc.noarch


Steps performed:
----------------
1. Create product (name- Containers) and repository (type- docker)-
# hammer product create --name Containers --organization "Default Organization"

# hammer repository create --name "rhel8/toolbox" --docker-upstream-name "rhel8/toolbox" --content-type docker --url https://registry.redhat.io --upstream-username <kerberos username> --upstream-password <kerberos password> --docker-tags-whitelist "8.7-5" --download-policy immediate --product Containers --organization "Default Organization"


2. Sync status from Sat UI-

Added Docker tags: 1
Total steps: 32/32
--------------------------------
Associating Content: 18/18
Downloading Artifacts: 12/12
Downloading tag list: 1/1
Processing Tags: 1/1
Un-Associating Content: 0/0


3. Delete the artifacts-
# du -sh /var/lib/pulp/media/artifact/
556M	/var/lib/pulp/media/artifact/ 

# rm -rf /var/lib/pulp/media/artifact/*


4. Go to Satellite UI Content --> Products --> Containers --> Repositories --> rhel8/toolbox --> Select Action dropdown --> Sync the repo
Sync failed but was able to get some data in pulp

# file /var/lib/pulp/media/artifact/*/*
/var/lib/pulp/media/artifact/48/fd57131d1a3ccda35d72ca802a44ced559941d9fe5ee160dbe3df0414e7cb3: ASCII text 


5. Go to Satellite UI Content --> Products --> Containers --> Repositories --> rhel8/toolbox --> Select Action dropdown --> Verify Content Checksum
Verify Content Checksum from Satellite web-UI or Invoke the Repair API

# curl --cert /etc/foreman/client_cert.pem --key /etc/foreman/client_key.pem -H "Content-Type: application/json" -X POST https://`hostname -f`/pulp/api/v3/repair/ | json_reformat
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    67  100    67    0     0   1488      0 --:--:-- --:--:-- --:--:--  1488
{
    "task": "/pulp/api/v3/tasks/60247332-91bd-4ad6-8225-dccf328c2678/"
}
 
# curl --cert /etc/foreman/client_cert.pem --key /etc/foreman/client_key.pem -H "Content-Type: application/json" -X GET https://`hostname -f`/pulp/api/v3/tasks/60247332-91bd-4ad6-8225-dccf328c2678/ | json_reformat
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   870  100   870    0     0  21219      0 --:--:-- --:--:-- --:--:-- 21219
{
    "pulp_href": "/pulp/api/v3/tasks/60247332-91bd-4ad6-8225-dccf328c2678/",
    "pulp_created": "2023-07-06T10:16:46.862202Z",
    "state": "running",
    "name": "pulpcore.app.tasks.repository.repair_all_artifacts",
    "logging_cid": "d79f26f447774f7eb33eef48f3552ddc",
    "started_at": "2023-07-06T10:16:46.888009Z",
    "finished_at": null,
    "error": null,
    "worker": "/pulp/api/v3/workers/cdf1d423-79a1-42f0-84a3-321e41801e93/",
    "parent_task": null,
    "child_tasks": [

    ],
    "task_group": null,
    "progress_reports": [
        {
            "message": "Identify missing units",
            "code": "repair.missing",
            "state": "running",
            "total": null,
            "done": 16,
            "suffix": null
        },
        {
            "message": "Identify corrupted units",
            "code": "repair.corrupted",
            "state": "running",
            "total": null,
            "done": 0,
            "suffix": null
        },
        {
            "message": "Repair corrupted units",
            "code": "repair.repaired",
            "state": "running",
            "total": null,
            "done": 11,
            "suffix": null
        }
    ],
    "created_resources": [

    ],
    "reserved_resources_record": [

    ]
}


Observation:
------------
Repair API is able to download files from Upstream

# file /var/lib/pulp/media/artifact/*/*
/var/lib/pulp/media/artifact/01/65232f07e945f1c1831eebb8c42e5695f80d47d7c3d787af20752c89333276: ASCII text, with very long lines, with no line terminators
/var/lib/pulp/media/artifact/48/fd57131d1a3ccda35d72ca802a44ced559941d9fe5ee160dbe3df0414e7cb3: ASCII text
/var/lib/pulp/media/artifact/5d/88a016637781e6d3eeb22c15c788ce1cc572e2dbb7bad4935d37a775c252d9: gzip compressed data, original size 169271808
/var/lib/pulp/media/artifact/62/9545c824752b6d75f7f22a73b6c7fcba9413b6017f2f67f58494a958bcd4ce: ASCII text, with very long lines, with no line terminators
/var/lib/pulp/media/artifact/64/554374912209545e580ebabc0c025363b6982d1fbffb00c50edfeea18e79b9: gzip compressed data, original size 212109824
/var/lib/pulp/media/artifact/6d/2cf55f7908bf17bd221297c8cff9750fad3187c3eb102d1c3f72a208efbf0b: gzip compressed data, original size 235748864
/var/lib/pulp/media/artifact/70/2f3a622329ec2d54c1bf3c13fa094c422d48abaad700a2e055c3b77bea15c0: UTF-8 Unicode text, with very long lines, with no line terminators
/var/lib/pulp/media/artifact/7f/060f344d579611663e84ea94d89d9372d7431e3e56e11a19e704207d5576e9: ASCII text, with very long lines, with no line terminators
/var/lib/pulp/media/artifact/9a/394c72313c9bcac63dc8980018582d90ece7ae65d03940e6d7e9b8d4cb884d: gzip compressed data, original size 264029696
/var/lib/pulp/media/artifact/a5/d805d6872a1693a518e623b95182f115915c37671f70c1b13eb136e930bfe4: gzip compressed data, original size 204123648
/var/lib/pulp/media/artifact/ac/88d64d7b415280e8c1221725a8455917b99711b754a5981d3460fa9fdf40f9: UTF-8 Unicode text, with very long lines, with no line terminators
/var/lib/pulp/media/artifact/b1/45dc70f6c036069d549762f39055162cac12b414e9e502fd0d4b3331a0f5eb: gzip compressed data, original size 171049472
/var/lib/pulp/media/artifact/b9/2727ef7443d73bb77323e5a969bf02187286fccf53930e83dc82be60a36529: gzip compressed data, original size 214139392
/var/lib/pulp/media/artifact/c6/25ce0354e96c43d4bfca9cb0a3ed8373b0707980f152582f731e61808df9bd: ASCII text, with very long lines, with no line terminators
/var/lib/pulp/media/artifact/d0/898311a8acc605593edbc1f37171f42aab926fffd820a0f0460e062a4f6717: UTF-8 Unicode text, with very long lines, with no line terminators
/var/lib/pulp/media/artifact/d4/d59e4eb38ff5eda9a9f6255575abc64501adb78153f5788bf580b1034d4f8d: gzip compressed data, original size 186984960
/var/lib/pulp/media/artifact/d5/2583c79ff512b040d97ca0ce70b61b9c496e674b789f8857efbcf93727e4b0: UTF-8 Unicode text, with very long lines, with no line terminators
Comment 10 errata-xmlrpc 2023-11-08 14:18:33 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.14 security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:6818
Note You need to log in before you can comment on or make changes to this bug.