Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 2043933

Summary: The pulp2-pulp3 migration should fail if not all the errata content has been migrated while upgrading to Satellite 6.10
Product: Red Hat Satellite Reporter: Sayan Das <saydas>
Component: RepositoriesAssignee: Ian Ballou <iballou>
Status: CLOSED ERRATA QA Contact: Sam Bible <sbible>
Severity: high Docs Contact:
Priority: high    
Version: 6.9.7CC: ahumbe, hlawatschek, jsherril, juwatts, osousa
Target Milestone: 6.9.9Keywords: PrioBumpGSS, Triaged, Upgrades
Target Release: Unused   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: tfm-rubygem-katello-3.18.1.51-1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-04-20 20:34:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sayan Das 2022-01-22 15:16:57 UTC
Description of problem:

Whether manual attempt or via satellite-maintain itself, The "Performing final content migration before switching content" step always succeeds even if some errata content is not migrated and then when "katello:pulp3_content_switchover" happens, It will encounter https://bugzilla.redhat.com/show_bug.cgi?id=2038241 .



Version-Release number of selected component (if applicable): 

6.9.z


How reproducible: 

Customer-specific


Steps to Reproduce:

Configure a Sat 6.9 and

1. satellite-maintain prep-6.10-upgrade
2. satellite-maintain content prepare
3. satellite-maintain upgrade run --target-version 6.10


Actual results:

The output of step 2:

# satellite-maintain content migration-stats
Running Retrieve Pulp 2 to Pulp 3 migration statistics
================================================================================
Retrieve Pulp 2 to Pulp 3 migration statistics:
============Migration Summary================
Migrated/Total RPMs: 132353/132353
Migrated/Total errata: 194411/194944
Migrated/Total repositories: 288/288
Estimated migration time based on yum content: 5 hours, 15 minutes

Note: ensure there is sufficient storage space for /var/lib/pulp/published to double in size before starting the migration process.
Check the size of /var/lib/pulp/published with 'du -sh /var/lib/pulp/published/'

Note: ensure there is sufficient storage space for postgresql.
You will need additional space for your postgresql database.  The partition holding '/var/opt/rh/rh-postgresql12/lib/pgsql/data/'
   will need additional free space equivalent to the size of your Mongo db database (/var/lib/mongodb/).
                                                                      [OK]
--------------------------------------------------------------------------------


--> 533 ERRATA not migrated but still, the command does not complain about it. 


Failure at Step 3:

Switching specified content over to pulp 3                            [FAIL]
Failed executing foreman-rake katello:pulp3_content_switchover, exit status 1:
 ERROR: at least one Erratum record has migrated_pulp3_href NULL value


Expected results:

The "satellite-maintain content prepare" or the underlying rake i.e. "foreman-rake katello:pulp3_migration" should fail if not all the ERRATA contents are migrated and should not proceed further with content-switchover phase.

The way this migration process reports about MISSING\CORRUPTED content, The rake\command should also report for UNMIGRATED ERRATA as well.


Additional info:

Even better if we get one of these outputs grabbed somewhere. 

# echo "select katello_repositories.id, katello_repositories.relative_path,count(*) from katello_repository_errata \
>    left join katello_repositories on katello_repositories.id = katello_repository_errata.repository_id \
>    where erratum_pulp3_href is null group by katello_repositories.id;" | su - postgres -c "psql foreman"
 id |                                          relative_path                                          | count
----+-------------------------------------------------------------------------------------------------+-------
 40 | PFG_Pilot/Pilot_Wave/Midrange_RHEL_7_x86_64/content/dist/rhel/server/7/7Server/x86_64/extras/os |   533
(1 row)


# echo "Katello::RepositoryErratum.where(erratum_pulp3_href: nil).pluck(:repository_id).uniq" | foreman-rake console

[40]

Comment 1 Justin Sherrill 2022-02-15 19:48:57 UTC
moving to 6.9.z

Comment 2 Ian Ballou 2022-02-21 20:51:24 UTC
Created redmine issue https://projects.theforeman.org/issues/34483 from this bug

Comment 3 Bryan Kearney 2022-02-22 00:05:02 UTC
Upstream bug assigned to iballou

Comment 4 Bryan Kearney 2022-02-22 00:05:05 UTC
Upstream bug assigned to iballou

Comment 5 Bryan Kearney 2022-02-23 00:05:14 UTC
Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/34483 has been resolved.

Comment 8 Sam Bible 2022-04-12 15:03:54 UTC
Version

6.9.9 - 2

Steps to Reproduce:

Configure a Satellite 6.9.9, add some repository content then upgrade.

Expected Results: Upgrade successful

Actual results: Upgrade successful 

output of satellite-maintain content migration-stats

Running Retrieve Pulp 2 to Pulp 3 migration statistics
================================================================================
Retrieve Pulp 2 to Pulp 3 migration statistics: 
============Migration Summary================
Migrated/Total RPMs: 789/789
Migrated/Total errata: 134/134
Migrated/Total repositories: 19/19
All content has been migrated.

Comment 12 errata-xmlrpc 2022-04-20 20:34:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.9.9 Async Bug Fix Update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:1478