Bug 1375075 - Repair repositories fail with 400 bad request in satellite 6.2
Summary: Repair repositories fail with 400 bad request in satellite 6.2
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Repositories
Version: 6.2.0
Hardware: x86_64
OS: Linux
high
high with 1 vote
Target Milestone: Unspecified
Assignee: Justin Sherrill
QA Contact: jcallaha
URL:
Whiteboard:
: 1381642 1438953 (view as bug list)
Depends On:
Blocks: 1425437 1446729
TreeView+ depends on / blocked
 
Reported: 2016-09-12 06:10 UTC by Amit Karsale
Modified: 2020-08-13 08:35 UTC (History)
14 users (show)

Fixed In Version: rubygem-katello-3.0.0.134-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1446727 (view as bug list)
Environment:
Last Closed: 2017-06-20 17:21:15 UTC
Target Upstream Version:


Attachments (Terms of Use)
verification screenshot 1 (53.81 KB, image/png)
2017-05-30 15:12 UTC, jcallaha
no flags Details
verification screenshot 2 (55.77 KB, image/png)
2017-05-30 15:13 UTC, jcallaha
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 17351 0 Normal Closed provide way to correct repos missing pulp 2021-01-06 18:51:21 UTC
Foreman Issue Tracker 18793 0 Normal Closed Content Views Create/Promotion Fails with "undefined method '[]' for nil:NilClass (NoMethodError) 2021-01-06 18:51:24 UTC
Foreman Issue Tracker 18816 0 High Closed RedHat Content sync stops working after upload of manifest 2021-01-06 18:51:23 UTC
Red Hat Product Errata RHBA-2017:1553 0 normal SHIPPED_LIVE Satellite 6.2.10 Async Bug Release 2017-06-20 21:19:07 UTC

Description Amit Karsale 2016-09-12 06:10:06 UTC
Description of problem:

On running the below command: 

#foreman-rake katello:generate_pulp_repositories

using the `repair_repos.rake` fail on Satellite 6.2 with following error message.

"Unable to fix repository with the pulp_id 'PAG-Deutschland-Red_Hat_Enterprise_Linux_Server-Red_Hat_Satellite_Tools_6_2_for_RHEL_7_Server_RPMs_x86_64'. Error => [400 Bad Request]"

The rake is used in the following scenario:

https://access.redhat.com/solutions/1453313


Version-Release number of selected component (if applicable): Satellite 6.2


How reproducible:

Steps to Reproduce:
1. Run the rake task `foreman-rake katello:generate_pulp_repositories
` to repair the broken repositories in satellite 6.2.


Actual results:
The task is throwing an error with 400 Bad Request

Expected results:
The rake should result in repair the broken repository.

Additional info:

To my findings this is a pulp error from pulp logs:
localhost pulp: pulp.server.webservices.middleware.exception:INFO: Invalid properties: ['distributor_type_id'] which is not being handled.



[SFDC ticket id: 01698598]

Comment 1 Pavel Moravec 2016-09-12 06:52:18 UTC
Relevant backtrace:

Sep  6 15:31:40 localhost pulp: pulp.server.controllers.repository:ERROR: (2800-65952) Exception adding distributor to repo [Obfuscated-Red_Hat_Enterprise_Linux_Server-Red_Hat_Satellite_Tools_6_2_for_RHEL_7_Server_RPMs_x86_64]; the repo will be deleted
Sep  6 15:31:40 localhost pulp: pulp.server.controllers.repository:ERROR: (2800-65952) Traceback (most recent call last):
Sep  6 15:31:40 localhost pulp: pulp.server.controllers.repository:ERROR: (2800-65952)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 425, in create_repo
Sep  6 15:31:40 localhost pulp: pulp.server.controllers.repository:ERROR: (2800-65952)     dist_controller.add_distributor(repo_id, type_id, plugin_config, auto_publish, dist_id)
Sep  6 15:31:40 localhost pulp: pulp.server.controllers.repository:ERROR: (2800-65952)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/distributor.py", line 50, in add_distributor
Sep  6 15:31:40 localhost pulp: pulp.server.controllers.repository:ERROR: (2800-65952)     raise exceptions.InvalidValue(['distributor_type_id'])
Sep  6 15:31:40 localhost pulp: pulp.server.controllers.repository:ERROR: (2800-65952) InvalidValue: Invalid properties: ['distributor_type_id']

I.e. repo_distributors mongo collection seems to have node distributor from 6.1, while running 6.2 that dropped that type of distributor.

Worth having that mongo collection to investigate more..

Comment 7 Justin Sherrill 2016-11-15 19:24:28 UTC
Connecting redmine issue http://projects.theforeman.org/issues/17351 from this bug

Comment 8 Bryan Kearney 2016-11-30 17:21:37 UTC
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/17351 has been resolved.

Comment 9 Justin Sherrill 2017-03-07 16:04:41 UTC
Adding a couple of issues found upstream because of this change:  18816 & 18793

Comment 10 Brad Buckingham 2017-04-18 20:05:13 UTC
*** Bug 1438953 has been marked as a duplicate of this bug. ***

Comment 11 jcallaha 2017-05-30 15:11:52 UTC

The single rake task mentioned in this bug report has been split into four, each with a specific focus

1. foreman-rake katello:correct_repositories

-bash-4.2# foreman-rake console
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_theme_satellite-0.1.43/app/models/concerns/satellite_packages.rb:4: warning: already initialized constant Katello::Ping::PACKAGES
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.134/app/models/katello/ping.rb:7: warning: previous definition of PACKAGES was here
/usr/share/foreman/lib/tasks/console.rake:4: warning: already initialized constant ARGV
Loading production environment (Rails 4.1.5)
irb(main):001:0> 
irb(main):005:0* Katello.pulp_server.resources.repository.delete(Katello::Repository.find(3).pulp_id)
=> {"spawned_tasks"=>[{"_href"=>"/pulp/api/v2/tasks/03e6af38-2292-4662-bde8-99f3e67fd119/", "task_id"=>"03e6af38-2292-4662-bde8-99f3e67fd119"}], "result"=>nil, "error"=>nil}
irb(main):006:0> 
irb(main):008:0* exit
-bash-4.2# 
-bash-4.2# foreman-rake katello:correct_repositories
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_theme_satellite-0.1.43/app/models/concerns/satellite_packages.rb:4: warning: already initialized constant Katello::Ping::PACKAGES
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.134/app/models/katello/ping.rb:7: warning: previous definition of PACKAGES was here
All operations will be skipped.  Re-run with COMMIT=true to perform corrections.
Processing Repository 1/29: Red Hat Satellite Tools 6.2 for RHEL 7 Server RPMs x86_64 (1)
Processing Repository 2/29: Red Hat Satellite Tools 6.2 for RHEL 6 Server RPMs x86_64 (2)
Processing Repository 3/29: Red Hat Enterprise Linux 7 Server - Supplementary RPMs x86_64 7Server (3)
Repository 3 Missing
Recreating 3
Processing Repository 4/29: Red Hat Enterprise Linux 7 Server - RH Common RPMs x86_64 7Server (4)
Processing Repository 5/29: Red Hat Enterprise Linux 7 Server - Optional RPMs x86_64 7Server (5)
Processing Repository 6/29: Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server (6)
Processing Repository 7/29: Red Hat Satellite 6.2 for RHEL 6 Server RPMs x86_64 (7)
Processing Repository 8/29: Red Hat Satellite 6.2 for RHEL 7 Server RPMs x86_64 (8)
Processing Repository 9/29: Red Hat Satellite Capsule 6.2 for RHEL 6 Server RPMs x86_64 (9)
Processing Repository 10/29: Red Hat Satellite Capsule 6.2 for RHEL 7 Server RPMs x86_64 (10)
Processing Repository 11/29: Red Hat Enterprise Linux 7 Server Kickstart x86_64 7.3 (11)
Processing Repository 12/29: Red Hat Software Collections RPMs for Red Hat Enterprise Linux 7 Server x86_64 7Server (12)
Processing Repository 13/29: Red Hat Software Collections RPMs for Red Hat Enterprise Linux 6 Server x86_64 6Server (13)
Processing Repository 14/29: Red Hat Satellite Tools 6.2 for RHEL 7 Server RPMs x86_64 (14)
Processing Repository 15/29: Red Hat Satellite Tools 6.2 for RHEL 7 Server RPMs x86_64 (15)
Processing Repository 16/29: Red Hat Enterprise Linux 7 Server - Supplementary RPMs x86_64 7Server (16)
Processing Repository 17/29: Red Hat Enterprise Linux 7 Server - Supplementary RPMs x86_64 7Server (17)
Processing Repository 18/29: Red Hat Enterprise Linux 7 Server - RH Common RPMs x86_64 7Server (18)
Processing Repository 19/29: Red Hat Enterprise Linux 7 Server - RH Common RPMs x86_64 7Server (19)
Processing Repository 20/29: Red Hat Enterprise Linux 7 Server - Optional RPMs x86_64 7Server (20)
Processing Repository 21/29: Red Hat Enterprise Linux 7 Server - Optional RPMs x86_64 7Server (21)
Processing Repository 22/29: Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server (22)
Processing Repository 23/29: Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server (23)
Processing Repository 24/29: Red Hat Software Collections RPMs for Red Hat Enterprise Linux 7 Server x86_64 7Server (24)
Processing Repository 25/29: Red Hat Software Collections RPMs for Red Hat Enterprise Linux 7 Server x86_64 7Server (25)
Processing Repository 26/29: ch-d (26)
Processing Repository 27/29: rizza (27)
Processing Repository 28/29: auto-tools (28)
Processing Repository 29/29: cloudbot (29)


2. foreman-rake katello:correct_puppet_environments

-bash-4.2# foreman-rake katello:correct_puppet_environments
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_theme_satellite-0.1.43/app/models/concerns/satellite_packages.rb:4: warning: already initialized constant Katello::Ping::PACKAGES
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.134/app/models/katello/ping.rb:7: warning: previous definition of PACKAGES was here
All operations will be skipped.  Re-run with COMMIT=true to perform corrections.
Processing Puppet Environment 1/3: Default_Organization-rhel7-1_0 (1)
Processing Puppet Environment 2/3: Default_Organization-puppet-1_0 (2)
Processing Puppet Environment 3/3: Default_Organization-Library-puppet (3)
Content View Puppet Environment 3 Missing, Creating.


3. foreman-rake katello:regenerate_repo_metadata

-bash-4.2# foreman-rake katello:regenerate_repo_metadata
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_theme_satellite-0.1.43/app/models/concerns/satellite_packages.rb:4: warning: already initialized constant Katello::Ping::PACKAGES
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.134/app/models/katello/ping.rb:7: warning: previous definition of PACKAGES was here
Regenerating 31 repositories.  You can monitor these on task id 2ab09eef-f4d9-4ac6-a780-10a7ce1d81e4

See attached screenshot for completed task.


4. foreman-rake katello:refresh_pulp_repo_details

-bash-4.2# foreman-rake katello:refresh_pulp_repo_details
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_theme_satellite-0.1.43/app/models/concerns/satellite_packages.rb:4: warning: already initialized constant Katello::Ping::PACKAGES
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.135/app/models/katello/ping.rb:7: warning: previous definition of PACKAGES was here
Refreshing 28 repositories.  You can monitor these on task id d52bcc4a-cd2f-4ca3-a26f-dcade1d8a0a3
 
See attached screenshot for completed task.

Comment 12 jcallaha 2017-05-30 15:12:39 UTC
Created attachment 1283479 [details]
verification screenshot 1

Comment 13 jcallaha 2017-05-30 15:13:00 UTC
Created attachment 1283480 [details]
verification screenshot 2

Comment 14 Brad Buckingham 2017-06-02 13:29:15 UTC
*** Bug 1381642 has been marked as a duplicate of this bug. ***

Comment 16 errata-xmlrpc 2017-06-20 17:21:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1553


Note You need to log in before you can comment on or make changes to this bug.