Bug 2049195

Summary: correct_repositories rake not sufficient to remove some repos that don't exist in Pulp
Product: Red Hat Satellite Reporter: Paul Dudley <pdudley>
Component: RepositoriesAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED DUPLICATE QA Contact: Cole Higgins <chiggins>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.10.2CC: saydas
Target Milestone: Unspecified   
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-02-02 19:38:50 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Paul Dudley 2022-02-01 17:32:20 UTC
Customer tries to publish Content view after 6.10 migration, receives this error on one or more subtasks:
~~~
 1) Task identifier: 2b1f9707-77bd-4379-a029-ddb574a75790
    Task action:     Publish
    Task errors:     Error message: the server returns an error
    HTTP status code: 404
    Response headers: {"Date"=>"Mon, 31 Jan 2022 17:45:19 GMT", "Server"=>"gunicorn", "Content-Type"=>"application/json", "Vary"=>"Accept,Cookie", "Allow"=>"GET, PUT, PATCH, DELETE, HEAD, OPTIONS", "X-Frame-Options"=>"SAMEORIGIN", "Content-Length"=>"23", "Correlation-ID"=>"75b31093-271d-4224-a03d-9666c209e990", "Access-Control-Expose-Headers"=>"Correlation-ID", "Via"=>"1.1 satellite.example.com"}
    Response body: {"detail":"Not found."}
~~~

These are from api calls to Pulp, seen here:
~~~
sos_commands/logs/journalctl_--no-pager:Jan 31 13:00:49 satellite.example.com pulpcore-api[76007]: pulp [b6d31759-764d-4894-ad93-0df271982f81]: django.request:WARNING: Not Found: /pulp/api/v3/repositories/rpm/rpm/7fc6603f-3a73-42a9-98d2-a72679181db9/
sos_commands/logs/journalctl_--no-pager:Jan 30 23:53:28 satellite.example.com pulpcore-api[78718]: pulp [5afbd3c0-c5d2-4209-975b-1e94eb0346cc]: django.request:WARNING: Not Found: /pulp/api/v3/repositories/rpm/rpm/4d99fcef-e958-49bb-817e-3e74c0ae3c0c/
~~~

The repo is in foremandb but not in pulpcore. To try to address this the correct_repositories rake was used. In one case this found 289 repos to clean:
~~~
 ▲ ~/cases/case rg 'Missing|Deleting' rakeoutput.txt | tail -n4
Repository 2316 Missing
Deleting 2316
Repository 2319 Missing
Deleting 2319

 ▲ ~/cases/case rg Deleting rakeoutput.txt | wc
    289     578    4268
~~~

After properly cleaning the previous publish attempts, the CV is published again but receives the same error on the subtasks. To resolve the issue this rake was needed:
~~~
foreman-rake console << EORAKE
Katello::Pulp3::RepositoryReference.all.select do |ref|
  api = Katello::RepositoryTypeManager.repository_types[ref.root_repository.content_type].pulp3_api_class.new(SmartProxy.pulp_primary)
  api.repositories_api.read(ref.repository_href)  
rescue => e
   if e&.code == 404
     ref.destroy
   end
end
EORAKE
~~~

After this rake the CV published without error. Creating this BZ to double-check if the correct_repos rake needs adjustment to properly handle occasions where the above rake solves the problem of repos existing in foreman but not in pulpcore.

Comment 1 Sayan Das 2022-02-02 09:13:06 UTC
If I am not assuming wrong, this is related to https://bugzilla.redhat.com/show_bug.cgi?id=2047675 but the ask here is to fix the correct_repositories rake so that it can function properly.

Comment 2 Paul Dudley 2022-02-02 19:38:50 UTC

*** This bug has been marked as a duplicate of bug 2047675 ***