Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 2073481

Summary: The Pulp 2 to Pulp 3 migration fails with error "NoMethodError: undefined method `link?' for nil:NilClass" in Red Hat Satellite 6.9.8
Product: Red Hat Satellite Reporter: Sayan Das <saydas>
Component: Content ViewsAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED NOTABUG QA Contact: Satellite QE Team <sat-qe-bz-list>
Severity: high Docs Contact: Brian Angelica <bangelic>
Priority: high    
Version: 6.9.8CC: ahumbe, bangelic, jpasqual, kkinge, mkalyat, sajha
Target Milestone: UnspecifiedKeywords: Documentation, Triaged, Upgrades
Target Release: Unused   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-05-24 18:28:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sayan Das 2022-04-08 15:11:23 UTC
Description of problem:

The Pulp 2 to Pulp 3 migration fails with error "NoMethodError: undefined method `link?' for nil:NilClass" in Red Hat Satellite 6.9.8. 

If there is a CV version present that has a repo in it but the repo itself in that version does not have an archived copy, then this traceback\error appears. 




Version-Release number of selected component (if applicable):

Satellite 6.9 (6.9.8)


How reproducible:

In customers environment 


Steps to Reproduce:
1. Install and configure satellite 6.9 with some content 

2. Find out a way to delete the archived copies of any repos within a CV version published in the Library environment.

3. Perform pulp2 to pulp3 migration. 


Actual results:

From foreman-maintain.log

~~
Prepare content for Pulp 3:
Checking for valid Katello configuration.
Starting task.

2022-04-01 08:11:23 -0400: Content migration starting. These steps may take a while to complete. Refer to `foreman-maintain content migration-stats` for an estimate.Migration failed, You will want to investigate: https://satellite.example.com/foreman_tasks/tasks/fa9130c3-5660-46cb-82ea-a03389e27968
rake aborted!
ForemanTasks::TaskError: Task fa9130c3-5660-46cb-82ea-a03389e27968: NoMethodError: undefined method `link?' for nil:NilClass
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.50/lib/katello/tasks/pulp3_migration.rake:43:in `block (2 levels) in <top (required)>'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/exe/rake:27:in `<top (required)>'
Tasks: TOP => katello:pulp3_migration
(See full trace by running task with --trace)
                                                                     [FAIL]
Failed executing preserve_output=true foreman-rake katello:pulp3_migration, exit status 1
~~


from production.log

~~
2022-03-30T14:00:29 [I|bac|] Task {label: Actions::Pulp3::ContentMigration, id: fb52dbf5-c872-4429-8280-b8d32c34c8bf, execution_plan_id: 560a5ff9-9b39-4994-a0ea-b6f34cf8cbdf} state changed: planned 
2022-03-30T14:00:30 [I|bac|] Task {label: Actions::Pulp3::ContentMigration, id: fb52dbf5-c872-4429-8280-b8d32c34c8bf, execution_plan_id: 560a5ff9-9b39-4994-a0ea-b6f34cf8cbdf} state changed: running 
2022-03-30T14:00:30 [E|bac|] undefined method `link?' for nil:NilClass (NoMethodError)
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.50/app/services/katello/pulp3/migration_plan.rb:168:in `block in content_view_migration'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.50/app/services/katello/pulp3/migration_plan.rb:166:in `each'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.50/app/services/katello/pulp3/migration_plan.rb:166:in `content_view_migration'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.50/app/services/katello/pulp3/migration_plan.rb:123:in `block in content_view_migrations_for'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.50/app/services/katello/pulp3/migration_plan.rb:117:in `each'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.50/app/services/katello/pulp3/migration_plan.rb:117:in `content_view_migrations_for'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.50/app/services/katello/pulp3/migration_plan.rb:43:in `block in repository_migrations'
| /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation/delegation.rb:87:in `each'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.50/app/services/katello/pulp3/migration_plan.rb:40:in `repository_migrations'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.50/app/services/katello/pulp3/migration_plan.rb:24:in `block in generate_plugins'
| /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.50/app/services/katello/pulp3/migration_plan.rb:21:in `map'   
..
..
2022-03-30T14:00:30 [I|bac|] Task {label: Actions::Pulp3::ContentMigration, id: fb52dbf5-c872-4429-8280-b8d32c34c8bf, execution_plan_id: 560a5ff9-9b39-4994-a0ea-b6f34cf8cbdf} state changed: stopped  result: warning
2022-03-30T14:00:30 [I|bac|] Task {label: Actions::Pulp3::ContentMigration, id: fb52dbf5-c872-4429-8280-b8d32c34c8bf, execution_plan_id: 560a5ff9-9b39-4994-a0ea-b6f34cf8cbdf} state changed: stopped  result: warning
~~


Expected results:

The satellite should be able to handle this situation properly rather than failing at this stage. 


Additional info:


Running this rake gives us the culprit and if we can delete that version of CV and retry migration , that will work fine this time:

# echo -e '
Katello::ContentView.non_default.each do |cv|
 Katello::Repository.in_default_view.each do |library_instance|
   cv.versions.with_library_repo(library_instance).each do |version|
     if version.archived_repos.find_by(:library_instance_id => library_instance.id).nil?
       found = version.repositories.find_by(:library_instance_id => library_instance.id)
       puts "#{cv.name} #{version.version}: #{library_instance.name} : env repo: #{found.id} in #{found.environment&.name}"    
     end
   end
 end
end ; nil
' | foreman-rake console

Example:  RHEL6 24.0: Red Hat Enterprise Linux 6 Server RPMs x86_64 6.10 : env repo: 48 in Library


Perhaps: The error message can be made a bit more understandable and also "katello_correct_repositories" rake can be improved to be able to repair such scenarios. 

KB: https://access.redhat.com/solutions/6898881

Comment 4 Sayan Das 2023-04-11 14:34:49 UTC

I have been discussing this with Katello team and it seems the best option we can take right now is to mention https://access.redhat.com/solutions/6898881 as one of the mandatory pre-requisites in the following section of the migration doc :

i.e. under "Pre-Migration Checks" of "3.1.1. Preparing to Migrate Content to Pulp 3"

https://access.redhat.com/documentation/en-us/red_hat_satellite/6.10/html-single/upgrading_and_updating_red_hat_satellite/index#preparing_to_migrate_pulp_content