Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1665657

Summary:	Upgrade Step: katello:upgrades:3.11:update_puppet_repos failed during 6.4 to 6.5 upgrade
Product:	Red Hat Satellite	Reporter:	Nikhil Kathole <nkathole>
Component:	Repositories	Assignee:	Justin Sherrill <jsherril>
Status:	CLOSED ERRATA	QA Contact:	vijsingh
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	6.5.0	CC:	egolov, inecas, mbacovsk
Target Milestone:	6.5.0	Keywords:	Regression, Triaged, Upgrades
Target Release:	Unused
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	tfm-rubygem-katello-3.10.0.12-1	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-05-14 12:39:53 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1653153

Description Nikhil Kathole 2019-01-12 06:34:42 UTC

Description of problem:

Upgrade from 6.4 to 6.5 failed at Upgrade Step: katello:upgrades:3.11:update_puppet_repos

=============================================
Upgrade Step 1/7: katello:correct_repositories. This may take a long while.
Processing Repository 1/1: puppetrepo (1)
=============================================
Upgrade Step 2/7: katello:correct_puppet_environments. This may take a long while.
=============================================
Upgrade Step 3/7: katello:clean_backend_objects. This may take a long while.
0 orphaned consumer id(s) found in candlepin.
Candlepin orphaned consumers: []
0 orphaned consumer id(s) found in pulp.
Pulp orphaned consumers: []
=============================================
Upgrade Step 4/7: katello:upgrades:3.8:clear_checksum_type. =============================================
Upgrade Step 5/7: katello:upgrades:3.9:migrate_sync_plans. Starting recurring logic for migrated sync plans and deleting Pulp schedules
=============================================
Upgrade Step 6/7: katello:upgrades:3.10:clear_invalid_repo_credentials. =============================================
Upgrade Step 7/7: katello:upgrades:3.11:update_puppet_repos. Refreshing repository puppetrepo (1)
Failed upgrade task: katello:upgrades:3.11:update_puppet_repos, see logs for more information.
foreman-rake upgrade:run finished successfully!
Upgrade completed!


Version-Release number of selected component (if applicable): 
6.5.0 snap 11 


How reproducible:
always

Steps to Reproduce:
1. On satellite 6.4.1,

# hammer repository list
---|------------|-------------|--------------|--------------------------------------------
ID | NAME       | PRODUCT     | CONTENT TYPE | URL                                        
---|------------|-------------|--------------|--------------------------------------------
1  | puppetrepo | puppet repo | puppet       | http://omaciel.fedorapeople.org/bagoftricks
---|------------|-------------|--------------|--------------------------------------------
[root@qe-sat6-client-arch ~]# hammer repository info --id 1
ID:                 1
Name:               puppetrepo
Label:              puppetrepo
Organization:       Default Organization
Red Hat Repository: no
Content Type:       puppet
Mirror on Sync:     yes
URL:                http://omaciel.fedorapeople.org/bagoftricks
Publish Via HTTP:   yes
Published At:       http://qe-sat6-client-arch.usersys.redhat.com/pulp/puppet/7aa60922-5aaf-456a-9710-00ceb6b55bec/
Relative Path:      Default_Organization/Library/custom/puppet_repo/puppetrepo
Download Policy:    
Product:            
    ID:   1
    Name: puppet repo
GPG Key:            

Sync:               
    Status:         Success
    Last Sync Date: 39 minutes
Created:            2019/01/12 05:53:37
Updated:            2019/01/12 05:53:40
Content Counts:     
    Puppet Modules: 12


2. Enable 6.5 repos
3. foreman-maintain upgrade run --target-version 6.5


Actual results: Upgrade step  katello:upgrades:3.11:update_puppet_repos failed.


Expected results: Successful upgrade!


Additional info: 

# foreman-rake katello:upgrades:3.11:update_puppet_repos --trace
/usr/share/foreman/lib/foreman.rb:8: warning: already initialized constant Foreman::UUID_REGEXP
/usr/share/foreman/lib/foreman.rb:8: warning: previous definition of UUID_REGEXP was here
** Invoke katello:upgrades:3.11:update_puppet_repos (first_time)
** Invoke environment (first_time)
** Execute environment
/usr/share/foreman/lib/foreman.rb:8: warning: already initialized constant Foreman::UUID_REGEXP
/usr/share/foreman/lib/foreman.rb:8: warning: previous definition of UUID_REGEXP was here
** Invoke katello:upgrades:3.11:update_puppet_repos (first_time)
** Invoke environment (first_time)
** Execute environment
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/content_facet_host_extensions.rb:7: warning: already initialized constant Katello::Concerns::ContentFacetHostExtensions::ERRATA_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/content_facet_host_extensions.rb:7: warning: previous definition of ERRATA_STATUS_MAP was here
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/content_facet_host_extensions.rb:14: warning: already initialized constant Katello::Concerns::ContentFacetHostExtensions::TRACE_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/content_facet_host_extensions.rb:14: warning: previous definition of TRACE_STATUS_MAP was here
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:13: warning: already initialized constant Katello::Concerns::SubscriptionFacetHostExtensions::SUBSCRIPTION_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:13: warning: previous definition of SUBSCRIPTION_STATUS_MAP was here
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:21: warning: already initialized constant Katello::Concerns::SubscriptionFacetHostExtensions::SLA_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:21: warning: previous definition of SLA_STATUS_MAP was here
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:26: warning: already initialized constant Katello::Concerns::SubscriptionFacetHostExtensions::USAGE_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:26: warning: previous definition of USAGE_STATUS_MAP was here
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:31: warning: already initialized constant Katello::Concerns::SubscriptionFacetHostExtensions::ROLE_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:31: warning: previous definition of ROLE_STATUS_MAP was here
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:36: warning: already initialized constant Katello::Concerns::SubscriptionFacetHostExtensions::ADDONS_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:36: warning: previous definition of ADDONS_STATUS_MAP was here
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:41: warning: already initialized constant Katello::Concerns::SubscriptionFacetHostExtensions::PURPOSE_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:41: warning: previous definition of PURPOSE_STATUS_MAP was here
Refreshing repository puppetrepo (1)
rake aborted!
NoMethodError: undefined method `[]' for nil:NilClass
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/lib/katello/tasks/upgrades/3.11/update_puppet_repos.rake:10:in `block (5 levels) in <top (required)>'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/delegation.rb:41:in `each'
/opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activerecord-5.2.1/lib/active_record/relation/delegation.rb:41:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/lib/katello/tasks/upgrades/3.11/update_puppet_repos.rake:7:in `block (4 levels) in <top (required)>'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'
Tasks: TOP => katello:upgrades:3.11:update_puppet_repos
(See full trace by running task with --trace)
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/content_facet_host_extensions.rb:7: warning: already initialized constant Katello::Concerns::ContentFacetHostExtensions::ERRATA_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/content_facet_host_extensions.rb:7: warning: previous definition of ERRATA_STATUS_MAP was here
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/content_facet_host_extensions.rb:14: warning: already initialized constant Katello::Concerns::ContentFacetHostExtensions::TRACE_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/content_facet_host_extensions.rb:14: warning: previous definition of TRACE_STATUS_MAP was here
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:13: warning: already initialized constant Katello::Concerns::SubscriptionFacetHostExtensions::SUBSCRIPTION_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:13: warning: previous definition of SUBSCRIPTION_STATUS_MAP was here
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:21: warning: already initialized constant Katello::Concerns::SubscriptionFacetHostExtensions::SLA_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:21: warning: previous definition of SLA_STATUS_MAP was here
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:26: warning: already initialized constant Katello::Concerns::SubscriptionFacetHostExtensions::USAGE_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:26: warning: previous definition of USAGE_STATUS_MAP was here
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:31: warning: already initialized constant Katello::Concerns::SubscriptionFacetHostExtensions::ROLE_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:31: warning: previous definition of ROLE_STATUS_MAP was here
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:36: warning: already initialized constant Katello::Concerns::SubscriptionFacetHostExtensions::ADDONS_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:36: warning: previous definition of ADDONS_STATUS_MAP was here
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:41: warning: already initialized constant Katello::Concerns::SubscriptionFacetHostExtensions::PURPOSE_STATUS_MAP
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0.9/app/models/katello/concerns/subscription_facet_host_extensions.rb:41: warning: previous definition of PURPOSE_STATUS_MAP was here
** Execute katello:upgrades:3.11:update_puppet_repos
Refreshing repository puppetrepo (1)
rake aborted!

Comment 5 Justin Sherrill 2019-01-15 14:20:20 UTC

Created redmine issue https://projects.theforeman.org/issues/25866 from this bug

Comment 6 Bryan Kearney 2019-01-15 15:05:01 UTC

Upstream bug assigned to jsherril

Comment 7 Bryan Kearney 2019-01-15 15:05:03 UTC

Upstream bug assigned to jsherril

Comment 8 Bryan Kearney 2019-01-15 15:05:49 UTC

Upstream bug assigned to jsherril

Comment 9 Bryan Kearney 2019-01-15 15:05:50 UTC

Upstream bug assigned to jsherril

Comment 10 Bryan Kearney 2019-01-15 15:06:41 UTC

Upstream bug assigned to jsherril

Comment 11 Bryan Kearney 2019-01-15 15:06:42 UTC

Upstream bug assigned to jsherril

Comment 12 Bryan Kearney 2019-01-15 15:07:52 UTC

Upstream bug assigned to jsherril

Comment 13 Bryan Kearney 2019-01-15 15:07:54 UTC

Upstream bug assigned to jsherril

Comment 14 Bryan Kearney 2019-01-15 15:08:46 UTC

Upstream bug assigned to jsherril

Comment 15 Bryan Kearney 2019-01-15 15:08:47 UTC

Upstream bug assigned to jsherril

Comment 16 Bryan Kearney 2019-01-17 15:04:56 UTC

Moving this bug to POST for triage into Satellite 6 since the upstream issue https://projects.theforeman.org/issues/25866 has been resolved.

Comment 18 vijsingh 2019-01-22 07:21:08 UTC

Verified:

@Satellite 6.5.0 Snap 12.0

Automation Upgrade passed.

Comment 21 errata-xmlrpc 2019-05-14 12:39:53 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:1222