Bug 1978571

Summary:	Capsule sync failed after upgrade with missing repository type error.
Product:	Red Hat Satellite	Reporter:	Devendra Singh <desingh>
Component:	Repositories	Assignee:	Justin Sherrill <jsherril>
Status:	CLOSED ERRATA	QA Contact:	Danny Synk <dsynk>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	6.10.0	CC:	dsynk, ggainey, jsherril, pcreech, ttereshc
Target Milestone:	6.10.0	Keywords:	PrioBumpQA, Regression, Triaged, Upgrades
Target Release:	Unused
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-11-16 14:12:20 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Devendra Singh 2021-07-02 08:14:33 UTC

Description of problem: Capsule sync failed after upgrade with a missing repository type error.


Version-Release number of selected component (if applicable):
6.10 Snap7

How reproducible:
always

Steps to Reproduce:
1. Prepare the 6.9.z setup using the upgrade template.
2. Run the pulp migration.
3. Pulp migration and it completes successfully.
4. Run the upgrade from 6.9 to 6.10 snap7 and upgrade completed successfully with BZ#1967685
5- Prepare a 6.10 capsule setup and add that capsule with the upgraded satellite.
6- Assign the organization's life cycle environment and location to the capsule.
7- Run the capsule sync.
8- Capsule sync failed with "can not find repository type" error

Actual results:
Capsule sync failed with "can not find repository type" error

Expected results:
Capsule sync should be completed successfully.

Additional info:

Comment 10 Justin Sherrill 2021-07-19 15:50:53 UTC

Thank you! I think i see the issue.  
The upgrade task: 

foreman-rake  katello:upgrades:4.0:remove_ostree_puppet_content

failed with:

RestClient::Forbidden: Katello::Resources::Candlepin::Content: 403 Forbidden {"displayMessage":"content \"3825\" is locked","requestUuid":"193b2c63-b4ab-4562-b5a1-d59c24b845a5"} (DELETE /candlepin/owners/cMgwGMw/content/3825)
/opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/abstract_response.rb:223:in `exception_with_response'
/opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/abstract_response.rb:103:in `return!'
/opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:809:in `process_result'
/opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:725:in `block in transmit'
/opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:715:in `transmit'
/opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:145:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/request.rb:52:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.2/lib/restclient/resource.rb:94:in `delete'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.1/app/lib/katello/http_resource.rb:119:in `delete'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.1/app/lib/katello/resources/candlepin/content.rb:25:in `destroy'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.1/lib/katello/tasks/upgrades/4.0/remove_ostree_puppet_content.rake:8:in `block (5 levels) in <top (required)>'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/relation/delegation.rb:87:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/relation/delegation.rb:87:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.1.1/lib/katello/tasks/upgrades/4.0/remove_ostree_puppet_content.rake:7:in `block (4 levels) in <top (required)>'
/opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/exe/rake:27:in `<top (required)>'
Tasks: TOP => katello:upgrades:4.0:remove_ostree_puppet_content
(See full trace by running task with --trace)


It looks like this rake task isn't handling Red hat ostree content properly.

Comment 11 Justin Sherrill 2021-07-19 20:59:29 UTC

Created redmine issue https://projects.theforeman.org/issues/33058 from this bug

Comment 12 Bryan Kearney 2021-07-20 00:00:49 UTC

Upstream bug assigned to jsherril

Comment 13 Bryan Kearney 2021-07-20 00:00:52 UTC

Upstream bug assigned to jsherril

Comment 14 Bryan Kearney 2021-07-31 00:00:49 UTC

Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/33058 has been resolved.

Comment 15 Danny Synk 2021-10-19 19:11:42 UTC

Verified on Satellite 6.10, snap 23 (katello-4.1.1-2.el7sat.noarch).

Steps to Test:
1. Deploy Satellite 6.9.7, snap 1.
2. Enable and sync an OSTree repository to Satellite (rhel-atomic-host-rpms).
3. Migrate from Pulp 2 to Pulp 3.
4. Upgrade the Satellite from 6.9 to 6.10.
5. Synchronize another repository to the Satellite. 
5. Deploy a Capsule 6.10.
6. Configure the Capsule for communication with the upgraded Satellite.
7. Assign the lifecycle environment containing the repository to the Capsule.
8. Synchronize the Capsule.

Expected Results:
The OSTree repository is deleted from the Satellite during the upgrade to 6.10. Syncing other repositories to Capsule after the upgrade is successful.

Actual Results:
The OSTree repository is deleted from the Satellite during the upgrade to 6.10. Syncing other repositories to Capsule after the upgrade is successful.

Comment 18 errata-xmlrpc 2021-11-16 14:12:20 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Satellite 6.10 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:4702