Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1375075

Summary:

Repair repositories fail with 400 bad request in satellite 6.2

Product:

Red Hat Satellite

Reporter:

Amit Karsale <akarsale>

Component:

Repositories

Assignee:

Justin Sherrill <jsherril>

Status:

CLOSED ERRATA

QA Contact:

jcallaha

Severity:

high

Docs Contact:

Priority:

high

Version:

6.2.0

CC:

abalakht, aupadhye, bbuckingham, bkearney, brubisch, fgarciad, jcallaha, jentrena, mmccune, phess, pmoravec, sreber, vijsingh, zhunting

Target Milestone:

Unspecified

Keywords:

Triaged

Target Release:

Unused

Hardware:

x86_64

OS:

Linux

Whiteboard:

Fixed In Version:

rubygem-katello-3.0.0.134-1

Doc Type:

If docs needed, set a value

Doc Text:

Story Points:

---

Clone Of:

Clones:

1446727 (view as bug list)

Environment:

Last Closed:

2017-06-20 17:21:15 UTC

Type:

Bug

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Bug Depends On:

Bug Blocks:

1425437, 1446729

Attachments:

Description	Flags
verification screenshot 1	none
verification screenshot 2	none

Description Amit Karsale 2016-09-12 06:10:06 UTC

Description of problem:

On running the below command: 

#foreman-rake katello:generate_pulp_repositories

using the `repair_repos.rake` fail on Satellite 6.2 with following error message.

"Unable to fix repository with the pulp_id 'PAG-Deutschland-Red_Hat_Enterprise_Linux_Server-Red_Hat_Satellite_Tools_6_2_for_RHEL_7_Server_RPMs_x86_64'. Error => [400 Bad Request]"

The rake is used in the following scenario:

https://access.redhat.com/solutions/1453313


Version-Release number of selected component (if applicable): Satellite 6.2


How reproducible:

Steps to Reproduce:
1. Run the rake task `foreman-rake katello:generate_pulp_repositories
` to repair the broken repositories in satellite 6.2.


Actual results:
The task is throwing an error with 400 Bad Request

Expected results:
The rake should result in repair the broken repository.

Additional info:

To my findings this is a pulp error from pulp logs:
localhost pulp: pulp.server.webservices.middleware.exception:INFO: Invalid properties: ['distributor_type_id'] which is not being handled.



[SFDC ticket id: 01698598]

Comment 1 Pavel Moravec 2016-09-12 06:52:18 UTC

Relevant backtrace:

Sep  6 15:31:40 localhost pulp: pulp.server.controllers.repository:ERROR: (2800-65952) Exception adding distributor to repo [Obfuscated-Red_Hat_Enterprise_Linux_Server-Red_Hat_Satellite_Tools_6_2_for_RHEL_7_Server_RPMs_x86_64]; the repo will be deleted
Sep  6 15:31:40 localhost pulp: pulp.server.controllers.repository:ERROR: (2800-65952) Traceback (most recent call last):
Sep  6 15:31:40 localhost pulp: pulp.server.controllers.repository:ERROR: (2800-65952)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 425, in create_repo
Sep  6 15:31:40 localhost pulp: pulp.server.controllers.repository:ERROR: (2800-65952)     dist_controller.add_distributor(repo_id, type_id, plugin_config, auto_publish, dist_id)
Sep  6 15:31:40 localhost pulp: pulp.server.controllers.repository:ERROR: (2800-65952)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/distributor.py", line 50, in add_distributor
Sep  6 15:31:40 localhost pulp: pulp.server.controllers.repository:ERROR: (2800-65952)     raise exceptions.InvalidValue(['distributor_type_id'])
Sep  6 15:31:40 localhost pulp: pulp.server.controllers.repository:ERROR: (2800-65952) InvalidValue: Invalid properties: ['distributor_type_id']

I.e. repo_distributors mongo collection seems to have node distributor from 6.1, while running 6.2 that dropped that type of distributor.

Worth having that mongo collection to investigate more..

Comment 7 Justin Sherrill 2016-11-15 19:24:28 UTC

Connecting redmine issue http://projects.theforeman.org/issues/17351 from this bug

Comment 8 Bryan Kearney 2016-11-30 17:21:37 UTC

Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/17351 has been resolved.

Comment 9 Justin Sherrill 2017-03-07 16:04:41 UTC

Adding a couple of issues found upstream because of this change:  18816 & 18793

Comment 10 Brad Buckingham 2017-04-18 20:05:13 UTC

*** Bug 1438953 has been marked as a duplicate of this bug. ***

Comment 11 jcallaha 2017-05-30 15:11:52 UTC


The single rake task mentioned in this bug report has been split into four, each with a specific focus

1. foreman-rake katello:correct_repositories

-bash-4.2# foreman-rake console
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_theme_satellite-0.1.43/app/models/concerns/satellite_packages.rb:4: warning: already initialized constant Katello::Ping::PACKAGES
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.134/app/models/katello/ping.rb:7: warning: previous definition of PACKAGES was here
/usr/share/foreman/lib/tasks/console.rake:4: warning: already initialized constant ARGV
Loading production environment (Rails 4.1.5)
irb(main):001:0> 
irb(main):005:0* Katello.pulp_server.resources.repository.delete(Katello::Repository.find(3).pulp_id)
=> {"spawned_tasks"=>[{"_href"=>"/pulp/api/v2/tasks/03e6af38-2292-4662-bde8-99f3e67fd119/", "task_id"=>"03e6af38-2292-4662-bde8-99f3e67fd119"}], "result"=>nil, "error"=>nil}
irb(main):006:0> 
irb(main):008:0* exit
-bash-4.2# 
-bash-4.2# foreman-rake katello:correct_repositories
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_theme_satellite-0.1.43/app/models/concerns/satellite_packages.rb:4: warning: already initialized constant Katello::Ping::PACKAGES
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.134/app/models/katello/ping.rb:7: warning: previous definition of PACKAGES was here
All operations will be skipped.  Re-run with COMMIT=true to perform corrections.
Processing Repository 1/29: Red Hat Satellite Tools 6.2 for RHEL 7 Server RPMs x86_64 (1)
Processing Repository 2/29: Red Hat Satellite Tools 6.2 for RHEL 6 Server RPMs x86_64 (2)
Processing Repository 3/29: Red Hat Enterprise Linux 7 Server - Supplementary RPMs x86_64 7Server (3)
Repository 3 Missing
Recreating 3
Processing Repository 4/29: Red Hat Enterprise Linux 7 Server - RH Common RPMs x86_64 7Server (4)
Processing Repository 5/29: Red Hat Enterprise Linux 7 Server - Optional RPMs x86_64 7Server (5)
Processing Repository 6/29: Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server (6)
Processing Repository 7/29: Red Hat Satellite 6.2 for RHEL 6 Server RPMs x86_64 (7)
Processing Repository 8/29: Red Hat Satellite 6.2 for RHEL 7 Server RPMs x86_64 (8)
Processing Repository 9/29: Red Hat Satellite Capsule 6.2 for RHEL 6 Server RPMs x86_64 (9)
Processing Repository 10/29: Red Hat Satellite Capsule 6.2 for RHEL 7 Server RPMs x86_64 (10)
Processing Repository 11/29: Red Hat Enterprise Linux 7 Server Kickstart x86_64 7.3 (11)
Processing Repository 12/29: Red Hat Software Collections RPMs for Red Hat Enterprise Linux 7 Server x86_64 7Server (12)
Processing Repository 13/29: Red Hat Software Collections RPMs for Red Hat Enterprise Linux 6 Server x86_64 6Server (13)
Processing Repository 14/29: Red Hat Satellite Tools 6.2 for RHEL 7 Server RPMs x86_64 (14)
Processing Repository 15/29: Red Hat Satellite Tools 6.2 for RHEL 7 Server RPMs x86_64 (15)
Processing Repository 16/29: Red Hat Enterprise Linux 7 Server - Supplementary RPMs x86_64 7Server (16)
Processing Repository 17/29: Red Hat Enterprise Linux 7 Server - Supplementary RPMs x86_64 7Server (17)
Processing Repository 18/29: Red Hat Enterprise Linux 7 Server - RH Common RPMs x86_64 7Server (18)
Processing Repository 19/29: Red Hat Enterprise Linux 7 Server - RH Common RPMs x86_64 7Server (19)
Processing Repository 20/29: Red Hat Enterprise Linux 7 Server - Optional RPMs x86_64 7Server (20)
Processing Repository 21/29: Red Hat Enterprise Linux 7 Server - Optional RPMs x86_64 7Server (21)
Processing Repository 22/29: Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server (22)
Processing Repository 23/29: Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server (23)
Processing Repository 24/29: Red Hat Software Collections RPMs for Red Hat Enterprise Linux 7 Server x86_64 7Server (24)
Processing Repository 25/29: Red Hat Software Collections RPMs for Red Hat Enterprise Linux 7 Server x86_64 7Server (25)
Processing Repository 26/29: ch-d (26)
Processing Repository 27/29: rizza (27)
Processing Repository 28/29: auto-tools (28)
Processing Repository 29/29: cloudbot (29)


2. foreman-rake katello:correct_puppet_environments

-bash-4.2# foreman-rake katello:correct_puppet_environments
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_theme_satellite-0.1.43/app/models/concerns/satellite_packages.rb:4: warning: already initialized constant Katello::Ping::PACKAGES
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.134/app/models/katello/ping.rb:7: warning: previous definition of PACKAGES was here
All operations will be skipped.  Re-run with COMMIT=true to perform corrections.
Processing Puppet Environment 1/3: Default_Organization-rhel7-1_0 (1)
Processing Puppet Environment 2/3: Default_Organization-puppet-1_0 (2)
Processing Puppet Environment 3/3: Default_Organization-Library-puppet (3)
Content View Puppet Environment 3 Missing, Creating.


3. foreman-rake katello:regenerate_repo_metadata

-bash-4.2# foreman-rake katello:regenerate_repo_metadata
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_theme_satellite-0.1.43/app/models/concerns/satellite_packages.rb:4: warning: already initialized constant Katello::Ping::PACKAGES
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.134/app/models/katello/ping.rb:7: warning: previous definition of PACKAGES was here
Regenerating 31 repositories.  You can monitor these on task id 2ab09eef-f4d9-4ac6-a780-10a7ce1d81e4

See attached screenshot for completed task.


4. foreman-rake katello:refresh_pulp_repo_details

-bash-4.2# foreman-rake katello:refresh_pulp_repo_details
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_theme_satellite-0.1.43/app/models/concerns/satellite_packages.rb:4: warning: already initialized constant Katello::Ping::PACKAGES
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.0.0.135/app/models/katello/ping.rb:7: warning: previous definition of PACKAGES was here
Refreshing 28 repositories.  You can monitor these on task id d52bcc4a-cd2f-4ca3-a26f-dcade1d8a0a3
 
See attached screenshot for completed task.

Comment 12 jcallaha 2017-05-30 15:12:39 UTC

Created attachment 1283479 [details]
verification screenshot 1

Comment 13 jcallaha 2017-05-30 15:13:00 UTC

Created attachment 1283480 [details]
verification screenshot 2

Comment 14 Brad Buckingham 2017-06-02 13:29:15 UTC

*** Bug 1381642 has been marked as a duplicate of this bug. ***

Comment 16 errata-xmlrpc 2017-06-20 17:21:15 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1553