Bug 2043933

Summary: The pulp2-pulp3 migration should fail if not all the errata content has been migrated while upgrading to Satellite 6.10
Product: Red Hat Satellite Reporter: Sayan Das <saydas>
Component: RepositoriesAssignee: Ian Ballou <iballou>
Status: CLOSED ERRATA QA Contact: Sam Bible <sbible>
Severity: high Docs Contact:
Priority: high    
Version: 6.9.7CC: ahumbe, hlawatschek, jsherril, juwatts, osousa
Target Milestone: 6.9.9Keywords: PrioBumpGSS, Triaged, Upgrades
Target Release: Unused   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: tfm-rubygem-katello-3.18.1.51-1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-04-20 20:34:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sayan Das 2022-01-22 15:16:57 UTC
Description of problem:

Whether manual attempt or via satellite-maintain itself, The "Performing final content migration before switching content" step always succeeds even if some errata content is not migrated and then when "katello:pulp3_content_switchover" happens, It will encounter https://bugzilla.redhat.com/show_bug.cgi?id=2038241 .



Version-Release number of selected component (if applicable): 

6.9.z


How reproducible: 

Customer-specific


Steps to Reproduce:

Configure a Sat 6.9 and

1. satellite-maintain prep-6.10-upgrade
2. satellite-maintain content prepare
3. satellite-maintain upgrade run --target-version 6.10


Actual results:

The output of step 2:

# satellite-maintain content migration-stats
Running Retrieve Pulp 2 to Pulp 3 migration statistics
================================================================================
Retrieve Pulp 2 to Pulp 3 migration statistics:
============Migration Summary================
Migrated/Total RPMs: 132353/132353
Migrated/Total errata: 194411/194944
Migrated/Total repositories: 288/288
Estimated migration time based on yum content: 5 hours, 15 minutes

Note: ensure there is sufficient storage space for /var/lib/pulp/published to double in size before starting the migration process.
Check the size of /var/lib/pulp/published with 'du -sh /var/lib/pulp/published/'

Note: ensure there is sufficient storage space for postgresql.
You will need additional space for your postgresql database.  The partition holding '/var/opt/rh/rh-postgresql12/lib/pgsql/data/'
   will need additional free space equivalent to the size of your Mongo db database (/var/lib/mongodb/).
                                                                      [OK]
--------------------------------------------------------------------------------


--> 533 ERRATA not migrated but still, the command does not complain about it. 


Failure at Step 3:

Switching specified content over to pulp 3                            [FAIL]
Failed executing foreman-rake katello:pulp3_content_switchover, exit status 1:
 ERROR: at least one Erratum record has migrated_pulp3_href NULL value


Expected results:

The "satellite-maintain content prepare" or the underlying rake i.e. "foreman-rake katello:pulp3_migration" should fail if not all the ERRATA contents are migrated and should not proceed further with content-switchover phase.

The way this migration process reports about MISSING\CORRUPTED content, The rake\command should also report for UNMIGRATED ERRATA as well.


Additional info:

Even better if we get one of these outputs grabbed somewhere. 

# echo "select katello_repositories.id, katello_repositories.relative_path,count(*) from katello_repository_errata \
>    left join katello_repositories on katello_repositories.id = katello_repository_errata.repository_id \
>    where erratum_pulp3_href is null group by katello_repositories.id;" | su - postgres -c "psql foreman"
 id |                                          relative_path                                          | count
----+-------------------------------------------------------------------------------------------------+-------
 40 | PFG_Pilot/Pilot_Wave/Midrange_RHEL_7_x86_64/content/dist/rhel/server/7/7Server/x86_64/extras/os |   533
(1 row)


# echo "Katello::RepositoryErratum.where(erratum_pulp3_href: nil).pluck(:repository_id).uniq" | foreman-rake console

[40]

Comment 1 Justin Sherrill 2022-02-15 19:48:57 UTC
moving to 6.9.z

Comment 2 Ian Ballou 2022-02-21 20:51:24 UTC
Created redmine issue https://projects.theforeman.org/issues/34483 from this bug

Comment 3 Bryan Kearney 2022-02-22 00:05:02 UTC
Upstream bug assigned to iballou

Comment 4 Bryan Kearney 2022-02-22 00:05:05 UTC
Upstream bug assigned to iballou

Comment 5 Bryan Kearney 2022-02-23 00:05:14 UTC
Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/34483 has been resolved.

Comment 8 Sam Bible 2022-04-12 15:03:54 UTC
Version

6.9.9 - 2

Steps to Reproduce:

Configure a Satellite 6.9.9, add some repository content then upgrade.

Expected Results: Upgrade successful

Actual results: Upgrade successful 

output of satellite-maintain content migration-stats

Running Retrieve Pulp 2 to Pulp 3 migration statistics
================================================================================
Retrieve Pulp 2 to Pulp 3 migration statistics: 
============Migration Summary================
Migrated/Total RPMs: 789/789
Migrated/Total errata: 134/134
Migrated/Total repositories: 19/19
All content has been migrated.

Comment 12 errata-xmlrpc 2022-04-20 20:34:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.9.9 Async Bug Fix Update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:1478