Bug 1908101

Summary: foreman-rake katello:correct_repositories is failing to create the repositories.
Product: Red Hat Satellite Reporter: Rudnei Bertol Jr. <rbertolj>
Component: RepositoriesAssignee: Ian Ballou <iballou>
Status: CLOSED ERRATA QA Contact: Cole Higgins <chiggins>
Severity: high Docs Contact:
Priority: high    
Version: 6.8.0CC: ahumbe, avnkumar, egolov, iballou, jkrajice, jpasqual, jsenkyri, ktordeur, kupadhya, matti.koskimies, mfalz, mmccune, nlager, patalber, pcreech, pmoravec, sadas, saydas, smajumda, wpinheir, zhunting
Target Milestone: 6.10.0Keywords: PrioBumpGSS, Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1952605 1973334 (view as bug list) Environment:
Last Closed: 2021-11-16 14:09:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
tfm-rubygem-katello-3.16.0.26-3.HFRHBZ18748501908101.el7sat.noarch.rpm none

Description Rudnei Bertol Jr. 2020-12-15 20:23:59 UTC
Description of problem:

The command `foreman-rake katello:correct_repositories COMMIT=true` is failing to recreate the pulp repositories when for some reason is was removed from pulp


Version-Release number of selected component (if applicable):

Satellite 6.8+

How reproducible:

Delete a repo from the pulp and try to recreate using the rake command.

Steps to Reproduce:
1. Remove a pulp repository
2. Run the foreman-rake to recreate the missing repository
3.

Actual results:

The rake command is failing with the following error.

"""
The Dynflow world was not initialized yet. If your plugin uses it, make sure to call Rails.application.dynflow.require! in some initializer
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/rails.rb:75:in `world'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/lib/foreman_tasks.rb:23:in `trigger'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/lib/foreman_tasks.rb:29:in `block in trigger_task'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/lib/foreman_tasks.rb:49:in `block in rails_safe_trigger_task'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
/opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.1/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/lib/foreman_tasks.rb:48:in `rails_safe_trigger_task'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/lib/foreman_tasks.rb:27:in `trigger_task'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-2.0.2/lib/foreman_tasks.rb:58:in `sync_task'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.0.11/lib/katello/tasks/repository.rake:148:in `handle_missing_repo'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.0.11/lib/katello/tasks/repository.rake:65:in `block (3 levels) in <top (required)>'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.1/lib/active_record/relation/batches.rb:70:in `block (2 levels) in find_each'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.1/lib/active_record/relation/batches.rb:70:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.1/lib/active_record/relation/batches.rb:70:in `block in find_each'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.1/lib/active_record/relation/batches.rb:136:in `block in find_in_batches'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.1/lib/active_record/relation/batches.rb:238:in `block in in_batches'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.1/lib/active_record/relation/batches.rb:222:in `loop'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.1/lib/active_record/relation/batches.rb:222:in `in_batches'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.1/lib/active_record/relation/batches.rb:135:in `find_in_batches'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.1/lib/active_record/relation/batches.rb:69:in `find_each'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.0.11/lib/katello/tasks/repository.rake:62:in `with_index'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.16.0.11/lib/katello/tasks/repository.rake:62:in `block (2 levels) in <top (required)>'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'
Tasks: TOP => katello:correct_repositories
(See full trace by running task with --trace)
"""

Expected results:

The missing repository be created with the rake.

Additional info:

Comment 4 Brad Buckingham 2021-04-05 13:32:09 UTC
*** Bug 1945321 has been marked as a duplicate of this bug. ***

Comment 15 Mike McCune 2021-05-14 21:07:38 UTC
*** Satellite 6.8.6 Hotfix Available ***

We have a hotfix available that resolves this bugzilla as well as BZ# 1874850


1) Download tfm-rubygem-katello-3.16.0.26-3.HFRHBZ18748501908101.el7sat.noarch.rpm from this bugzilla to your Satellite

2) stop services:

satellite-maintain service stop

3) Install:

rpm -Uvh tfm-rubygem-katello-3.16.0.26-3.HFRHBZ18748501908101.el7sat.noarch.rpm

4) restart:

satellite-maintain service start

5) resume operations

Comment 16 Mike McCune 2021-05-14 21:08:14 UTC
Created attachment 1783342 [details]
tfm-rubygem-katello-3.16.0.26-3.HFRHBZ18748501908101.el7sat.noarch.rpm

Comment 20 Tasos Papaioannou 2021-06-04 20:21:50 UTC
The rake task is failing because of an HTTP 404 error from pulp:

****
1.) Enable and sync Red Hat Enterprise Linux 7 Server - Extras RPMs x86_64.
2.) Get pulp repo id:

# curl --cert /etc/pki/katello/certs/pulp-client.crt --key /etc/pki/katello/private/pulp-client.key https://satellite.example.com/pulp/api/v3/repositories/

{"count":1,"next":null,"previous":null,"results":[{"pulp_href":"/pulp/api/v3/repositories/rpm/rpm/9cce6b55-5f9f-49c5-8c25-9e74eabb1d32/","pulp_created":"2021-06-04T20:06:01.704118Z","versions_href":"/pulp/api/v3/repositories/rpm/rpm/9cce6b55-5f9f-49c5-8c25-9e74eabb1d32/versions/","pulp_labels":{},"latest_version_href":"/pulp/api/v3/repositories/rpm/rpm/9cce6b55-5f9f-49c5-8c25-9e74eabb1d32/versions/1/","name":"Red_Hat_Enterprise_Linux_7_Server_-_Extras_RPMs_x86_64-19467","description":null,"remote":null}]}

3.) Delete repo in pulp:

# curl -X DELETE --cert /etc/pki/katello/certs/pulp-client.crt --key /etc/pki/katello/private/pulp-client.key https://satellite.example.com/pulp/api/v3/repositories/rpm/rpm/9cce6b55-5f9f-49c5-8c25-9e74eabb1d32/

{"task":"/pulp/api/v3/tasks/b16efce7-e6ac-4c93-bbf0-3363ffe2c790/"}

# curl --cert /etc/pki/katello/certs/pulp-client.crt --key /etc/pki/katello/private/pulp-client.key https://satellite.example.com/pulp/api/v3/repositories/

{"count":0,"next":null,"previous":null,"results":[]}

4.) Run the rake command:

# foreman-rake katello:correct_repositories COMMIT=false
`/usr/share/foreman` is not writable.
Bundler will use `/tmp/bundler20210604-11677-155gxt11677' as your home directory temporarily.
'ErbParser' is ignored.
'RubyParser' is ignored.
All operations will be skipped.  Re-run with COMMIT=true to perform corrections.
Processing Repository 1/1: Red Hat Enterprise Linux 7 Server - Extras RPMs x86_64 (1)
Repository 1 Missing
Recreating 1

# foreman-rake katello:correct_repositories COMMIT=true
`/usr/share/foreman` is not writable.
Bundler will use `/tmp/bundler20210604-11721-1ejud0d11721' as your home directory temporarily.
'ErbParser' is ignored.
'RubyParser' is ignored.
Processing Repository 1/1: Red Hat Enterprise Linux 7 Server - Extras RPMs x86_64 (1)
Repository 1 Missing
Recreating 1
rake aborted!
ForemanTasks::TaskError: Task 41c6da36-c87b-400c-875b-4dbe5b824fe3: PulpRpmClient::ApiError: Error message: the server returns an error
HTTP status code: 404
Response headers: {"Date"=>"Fri, 04 Jun 2021 20:15:27 GMT", "Server"=>"gunicorn/20.0.4", "Content-Type"=>"application/json", "Vary"=>"Accept,Cookie", "Allow"=>"GET, PUT, PATCH, DELETE, HEAD, OPTIONS", "X-Frame-Options"=>"SAMEORIGIN", "Content-Length"=>"23", "Correlation-ID"=>"5c057fa98ad049649ab40c344ec66466", "Access-Control-Expose-Headers"=>"Correlation-ID", "Via"=>"1.1 satellite.example.com"}
Response body: {"detail":"Not found."}
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.1/lib/foreman_tasks.rb:59:in `block in sync_task'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.1/lib/foreman_tasks.rb:58:in `tap'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-4.1.1/lib/foreman_tasks.rb:58:in `sync_task'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.0.rc2.1/lib/katello/tasks/repository.rake:130:in `handle_missing_repo'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.0.rc2.1/lib/katello/tasks/repository.rake:63:in `block (3 levels) in <top (required)>'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.5/lib/active_record/relation/batches.rb:70:in `block (2 levels) in find_each'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.5/lib/active_record/relation/batches.rb:70:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.5/lib/active_record/relation/batches.rb:70:in `block in find_each'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.5/lib/active_record/relation/batches.rb:136:in `block in find_in_batches'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.5/lib/active_record/relation/batches.rb:238:in `block in in_batches'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.5/lib/active_record/relation/batches.rb:222:in `loop'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.5/lib/active_record/relation/batches.rb:222:in `in_batches'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.5/lib/active_record/relation/batches.rb:135:in `find_in_batches'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.5/lib/active_record/relation/batches.rb:69:in `find_each'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.0.rc2.1/lib/katello/tasks/repository.rake:60:in `with_index'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.0.rc2.1/lib/katello/tasks/repository.rake:60:in `block (2 levels) in <top (required)>'
/opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/exe/rake:27:in `<top (required)>'
Tasks: TOP => katello:correct_repositories
(See full trace by running task with --trace)
****

Comment 21 Bryan Kearney 2021-06-05 00:06:12 UTC
Upstream bug assigned to jsherril

Comment 23 Ian Ballou 2021-08-04 17:09:07 UTC
Created redmine issue https://projects.theforeman.org/issues/33212 from this bug

Comment 24 Bryan Kearney 2021-08-04 20:04:06 UTC
Upstream bug assigned to iballou

Comment 25 Bryan Kearney 2021-08-04 20:04:08 UTC
Upstream bug assigned to iballou

Comment 43 errata-xmlrpc 2021-11-16 14:09:38 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Satellite 6.10 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:4702