Bug 1996485

Summary: Actions::Katello::Repository::IndexContent (skipped) causing sync failures
Product: Red Hat Satellite Reporter: Taft Sanders <tasander>
Component: PulpAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED DUPLICATE QA Contact: Lai <ltran>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.10.0CC: ttereshc
Target Milestone: 6.10.0Keywords: Triaged
Target Release: Unused   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-10-01 14:29:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Taft Sanders 2021-08-22 22:40:08 UTC
Description of problem:
Syncs failing on step "Actions::Katello::Repository::IndexContent" with the vague error message of:
===================
Error:

PulpRpmClient::ApiError

Error message: the server returns an error
===================

Version-Release number of selected component (if applicable):
6.10.0 Snap 14

How reproducible:
Every fresh install with multiple large syncs

Steps to Reproduce:
1. Fresh build
2. Enable 15-20 repos (make sure to have large repos like rhel 5,6,7 server rpms, rhel8 appstream/baseos, etc)
3. Sync them all at the same time

Actual results:
Sync is incomplete

Expected results:
This step to retry if the API couldn't be reached at that time or to fail with a better error message.

Additional info:
With the error message so vague it's hard to tell what the cause of the error was. Looking at the apache logs we can see that the return was a 200:

10.12.214.18 - - [20/Aug/2021:17:56:36 -0400] "GET /pulp/api/v3/tasks/cb16b660-d83c-4e0f-8976-370cf6d3f652/ HTTP/1.1" 200 562 "-" "OpenAPI-Generator/3.14.1/ruby"

and the task on pulp shows completed:
# curl -sk --cert /etc/pki/katello/certs/pulp-client.crt --key /etc/pki/katello/private/pulp-client.key https://10.12.214.18/pulp/api/v3/tasks/cb16b660-d83c-4e0f-8976-370cf6d3f652/ | python -m json.tool
{
    "child_tasks": [],
    "created_resources": [],
    "error": null,
    "finished_at": "2021-08-20T21:55:22.807958Z",
    "logging_cid": "4d074e55-7d35-49cb-b61e-9af231a42bc0",
    "name": "pulpcore.app.tasks.base.general_update",
    "parent_task": null,
    "progress_reports": [],
    "pulp_created": "2021-08-20T21:55:22.319112Z",
    "pulp_href": "/pulp/api/v3/tasks/cb16b660-d83c-4e0f-8976-370cf6d3f652/",
    "reserved_resources_record": [
        "/api/v3/distributions/"
    ],
    "started_at": "2021-08-20T21:55:22.418929Z",
    "state": "completed",
    "task_group": null,
    "worker": "/pulp/api/v3/workers/94a5640f-b32d-472a-99d7-f1533be1a957/"
}


Attempts to resync the repository seem to allow the repository to complete with no issues. This appears to only happen when syncing large repositories initially.

Comment 2 Tanya Tereshchenko 2021-08-23 20:16:52 UTC
Likely a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1995920