Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1994217

Summary: satellite upgrade failing with foreman-rake katello:pulp3_migration
Product: Red Hat Satellite Reporter: Ram Nainsingh Tiruwa <ramsingh>
Component: UpgradesAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED NOTABUG QA Contact: Devendra Singh <desingh>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 6.10.0   
Target Milestone: Unspecified   
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-08-23 11:50:02 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Ram Nainsingh Tiruwa 2021-08-17 04:32:01 UTC 
Satellite upgrading failing with below error:

--------------------------------------------------------------------------------
Switch support for certain content from Pulp 2 to Pulp 3: 
Performing final content migration before switching content           [FAIL]
Failed executing foreman-rake katello:pulp3_migration, exit status 255:
 Checking for valid Katello configuraton.
Starting task.
2021-08-17 00:09:39 -0400: Importing migrated yum repositories: 1/18
Some corrupted or missing content found, run 'foreman-maintain content migration-stats' for more information.
--------------------------------------------------------------------------------

Red Hat Satellite 6.10

Below are the repository enabled

----------------------------------------------
hammer> repository list
---|-----------------------------------------------------------------|---------------------------------------|--------------|---------------------------------------------------------------------------------
ID | NAME                                                            | PRODUCT                               | CONTENT TYPE | URL                                                                             
---|-----------------------------------------------------------------|---------------------------------------|--------------|---------------------------------------------------------------------------------
9  | Red Hat Enterprise Linux 7 Server Kickstart x86_64 7.8          | Red Hat Enterprise Linux Server       | yum          | https://cdn.redhat.com/content/dist/rhel/server/7/7.8/x86_64/kickstart          
12 | Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server           | Red Hat Enterprise Linux Server       | yum          | https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os             
19 | Red Hat Enterprise Linux 7 Server - TUS RPMs x86_64 7.6         | Red Hat Enterprise Linux Server - TUS | yum          | https://cdn.redhat.com/content/tus/rhel/server/7/7.6/x86_64/os                  
11 | Red Hat Enterprise Linux 8 for x86_64 - AppStream Kickstart 8.3 | Red Hat Enterprise Linux for x86_64   | yum          | https://cdn.redhat.com/content/dist/rhel8/8.3/x86_64/appstream/kickstart        
14 | Red Hat Enterprise Linux 8 for x86_64 - AppStream RPMs 8        | Red Hat Enterprise Linux for x86_64   | yum          | https://cdn.redhat.com/content/dist/rhel8/8/x86_64/appstream/os                 
10 | Red Hat Enterprise Linux 8 for x86_64 - BaseOS Kickstart 8.3    | Red Hat Enterprise Linux for x86_64   | yum          | https://cdn.redhat.com/content/dist/rhel8/8.3/x86_64/baseos/kickstart           
15 | Red Hat Enterprise Linux 8 for x86_64 - BaseOS RPMs 8           | Red Hat Enterprise Linux for x86_64   | yum          | https://cdn.redhat.com/content/dist/rhel8/8/x86_64/baseos/os                    
13 | Red Hat Satellite Tools 6.9 for RHEL 7 Server RPMs x86_64       | Red Hat Enterprise Linux Server       | yum          | https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6....
16 | Red Hat Satellite Tools 6.9 for RHEL 8 x86_64 RPMs              | Red Hat Enterprise Linux for x86_64   | yum          | https://cdn.redhat.com/content/dist/layered/rhel8/x86_64/sat-tools/6.9/os       

---------------------------------------------------

su - postgres -c "psql foreman -c \"select id, name, download_policy from katello_root_repositories;\""
 id |                              name                               | download_policy 
----+-----------------------------------------------------------------+-----------------
  9 | Red Hat Enterprise Linux 7 Server Kickstart x86_64 7.8          | immediate
 11 | Red Hat Enterprise Linux 8 for x86_64 - AppStream Kickstart 8.3 | immediate
 10 | Red Hat Enterprise Linux 8 for x86_64 - BaseOS Kickstart 8.3    | immediate
 12 | Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server           | on_demand
 13 | Red Hat Satellite Tools 6.9 for RHEL 7 Server RPMs x86_64       | on_demand
 14 | Red Hat Enterprise Linux 8 for x86_64 - AppStream RPMs 8        | on_demand
 15 | Red Hat Enterprise Linux 8 for x86_64 - BaseOS RPMs 8           | on_demand
 16 | Red Hat Satellite Tools 6.9 for RHEL 8 x86_64 RPMs              | on_demand
 19 | Red Hat Enterprise Linux 7 Server - TUS RPMs x86_64 7.6         | on_demand


From above one of the repository is not synced (rhel8 appstream repository) is not synced.


foreman-maintain content migration-stats command is completed successfully.

foreman-maintain content migration-stats 

Running Retrieve Pulp 2 to Pulp 3 migration statistics
================================================================================
Retrieve Pulp 2 to Pulp 3 migration statistics: 
============Migration Summary================
Migrated/Total RPMs: 67388/67409
Migrated/Total errata: 7333/7333
Migrated/Total repositories: 17/17

Estimated migration time based on yum content: fewer than 5 minutes

Note: ensure there is sufficient storage space for /var/lib/pulp/published to double in size before starting the migration process.
Check the size of /var/lib/pulp/published with 'du -sh /var/lib/pulp/published/'

============Missing/Corrupted Content Summary================
WARNING: MISSING OR CORRUPTED CONTENT DETECTED
Corrupted or Missing Rpm: 21/67409
Corrupted or missing content has been detected, you can examine the list of content in /tmp/unmigratable_content-20210817-30268-1sybi16 and take action by either:
1. Performing a 'Verify Checksum' sync under Advanced Sync Options, let it complete, and re-running the migration
2. Deleting/disabling the affected repositories and running orphan cleanup (foreman-rake katello:delete_orphaned_content) and re-running the migration
3. Manually correcting files on the filesystem in /var/lib/pulp/content/ and re-running the migration
4. Mark currently corrupted or missing content as skipped (foreman-rake katello:approve_corrupted_migration_content).  This will skip migration of missing or corrupted content.
                                                                      [OK]
--------------------------------------------------------------------------------