Bug 1994217 - satellite upgrade failing with foreman-rake katello:pulp3_migration
Summary: satellite upgrade failing with foreman-rake katello:pulp3_migration
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Upgrades
Version: 6.10.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: Unspecified
Assignee: satellite6-bugs
QA Contact: Devendra Singh
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-08-17 04:32 UTC by Ram Nayan Singh
Modified: 2021-08-23 11:50 UTC (History)
0 users

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-08-23 11:50:02 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Ram Nayan Singh 2021-08-17 04:32:01 UTC
Satellite upgrading failing with below error:

--------------------------------------------------------------------------------
Switch support for certain content from Pulp 2 to Pulp 3: 
Performing final content migration before switching content           [FAIL]
Failed executing foreman-rake katello:pulp3_migration, exit status 255:
 Checking for valid Katello configuraton.
Starting task.
2021-08-17 00:09:39 -0400: Importing migrated yum repositories: 1/18
Some corrupted or missing content found, run 'foreman-maintain content migration-stats' for more information.
--------------------------------------------------------------------------------

Red Hat Satellite 6.10

Below are the repository enabled

----------------------------------------------
hammer> repository list
---|-----------------------------------------------------------------|---------------------------------------|--------------|---------------------------------------------------------------------------------
ID | NAME                                                            | PRODUCT                               | CONTENT TYPE | URL                                                                             
---|-----------------------------------------------------------------|---------------------------------------|--------------|---------------------------------------------------------------------------------
9  | Red Hat Enterprise Linux 7 Server Kickstart x86_64 7.8          | Red Hat Enterprise Linux Server       | yum          | https://cdn.redhat.com/content/dist/rhel/server/7/7.8/x86_64/kickstart          
12 | Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server           | Red Hat Enterprise Linux Server       | yum          | https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os             
19 | Red Hat Enterprise Linux 7 Server - TUS RPMs x86_64 7.6         | Red Hat Enterprise Linux Server - TUS | yum          | https://cdn.redhat.com/content/tus/rhel/server/7/7.6/x86_64/os                  
11 | Red Hat Enterprise Linux 8 for x86_64 - AppStream Kickstart 8.3 | Red Hat Enterprise Linux for x86_64   | yum          | https://cdn.redhat.com/content/dist/rhel8/8.3/x86_64/appstream/kickstart        
14 | Red Hat Enterprise Linux 8 for x86_64 - AppStream RPMs 8        | Red Hat Enterprise Linux for x86_64   | yum          | https://cdn.redhat.com/content/dist/rhel8/8/x86_64/appstream/os                 
10 | Red Hat Enterprise Linux 8 for x86_64 - BaseOS Kickstart 8.3    | Red Hat Enterprise Linux for x86_64   | yum          | https://cdn.redhat.com/content/dist/rhel8/8.3/x86_64/baseos/kickstart           
15 | Red Hat Enterprise Linux 8 for x86_64 - BaseOS RPMs 8           | Red Hat Enterprise Linux for x86_64   | yum          | https://cdn.redhat.com/content/dist/rhel8/8/x86_64/baseos/os                    
13 | Red Hat Satellite Tools 6.9 for RHEL 7 Server RPMs x86_64       | Red Hat Enterprise Linux Server       | yum          | https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/sat-tools/6....
16 | Red Hat Satellite Tools 6.9 for RHEL 8 x86_64 RPMs              | Red Hat Enterprise Linux for x86_64   | yum          | https://cdn.redhat.com/content/dist/layered/rhel8/x86_64/sat-tools/6.9/os       

---------------------------------------------------

su - postgres -c "psql foreman -c \"select id, name, download_policy from katello_root_repositories;\""
 id |                              name                               | download_policy 
----+-----------------------------------------------------------------+-----------------
  9 | Red Hat Enterprise Linux 7 Server Kickstart x86_64 7.8          | immediate
 11 | Red Hat Enterprise Linux 8 for x86_64 - AppStream Kickstart 8.3 | immediate
 10 | Red Hat Enterprise Linux 8 for x86_64 - BaseOS Kickstart 8.3    | immediate
 12 | Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server           | on_demand
 13 | Red Hat Satellite Tools 6.9 for RHEL 7 Server RPMs x86_64       | on_demand
 14 | Red Hat Enterprise Linux 8 for x86_64 - AppStream RPMs 8        | on_demand
 15 | Red Hat Enterprise Linux 8 for x86_64 - BaseOS RPMs 8           | on_demand
 16 | Red Hat Satellite Tools 6.9 for RHEL 8 x86_64 RPMs              | on_demand
 19 | Red Hat Enterprise Linux 7 Server - TUS RPMs x86_64 7.6         | on_demand


From above one of the repository is not synced (rhel8 appstream repository) is not synced.


foreman-maintain content migration-stats command is completed successfully.

foreman-maintain content migration-stats 

Running Retrieve Pulp 2 to Pulp 3 migration statistics
================================================================================
Retrieve Pulp 2 to Pulp 3 migration statistics: 
============Migration Summary================
Migrated/Total RPMs: 67388/67409
Migrated/Total errata: 7333/7333
Migrated/Total repositories: 17/17

Estimated migration time based on yum content: fewer than 5 minutes

Note: ensure there is sufficient storage space for /var/lib/pulp/published to double in size before starting the migration process.
Check the size of /var/lib/pulp/published with 'du -sh /var/lib/pulp/published/'

============Missing/Corrupted Content Summary================
WARNING: MISSING OR CORRUPTED CONTENT DETECTED
Corrupted or Missing Rpm: 21/67409
Corrupted or missing content has been detected, you can examine the list of content in /tmp/unmigratable_content-20210817-30268-1sybi16 and take action by either:
1. Performing a 'Verify Checksum' sync under Advanced Sync Options, let it complete, and re-running the migration
2. Deleting/disabling the affected repositories and running orphan cleanup (foreman-rake katello:delete_orphaned_content) and re-running the migration
3. Manually correcting files on the filesystem in /var/lib/pulp/content/ and re-running the migration
4. Mark currently corrupted or missing content as skipped (foreman-rake katello:approve_corrupted_migration_content).  This will skip migration of missing or corrupted content.
                                                                      [OK]
--------------------------------------------------------------------------------


Note You need to log in before you can comment on or make changes to this bug.