Bug 2134230

1. Proposed title of this feature request

[RFE] Log pulp3 content migration process

2. Who is the customer behind the request?

     Account: red hat (acct #540155)

     TAM customer: no

     CSM customer: no

     Strategic: no

3. What is the nature and description of the request?

I'd like to request that the content migration process be logged somewhere, ideally somewhere that would be picked up by an sosreport.

4. Why does the customer need this? (List the business requirements here)

Content premigration can take a very long time; usually hours, but on rare occasions even days.  The upgrade process also contains a content migration step that can take a long time if content premigration is skipped, or if the customer waits a week or two between premigration and upgrade, or if they've configured a certain workaround to prevent a "Cursor not found" error (KCS 6521251).  During this time, customers frequently ask "how can I see that something is happening?", but we don't have great answers for them.  Because they cannot see much happening on the screen, they often assume (or insist) that nothing is happening, even if it is.

5. How would the customer like to achieve this? (List the functional requirements here)

This foreman-rake script was recently used in a case to monitor the migration task:

# foreman-rake console << EORAKE
namespace :katello do
  task=ForemanTasks::Task.find_by(external_id: '<external_id>')
  wait = ::Foreman::Cast.to_bool(ENV['wait'] || 'true')
  preserve_output = ::Foreman::Cast.to_bool(ENV['preserve_output'])
  if wait
    clear_count = nil 
    until !task.pending? || task.paused?
      \$stdout.print("\r#{' ' * clear_count}\r") if clear_count && !preserve_output #clear the line before printing
      \$stdout.print("\n") if preserve_output
      message = "#{Time.now.to_s}: #{task.humanized[:output]}"
      clear_count = message.length + 1 
      \$stdout.print(message)
      \$stdout.flush
  
      sleep(10)
      task = ForemanTasks::Task.find(task.id)
      puts
    end
    if task.result == 'warning' || task.result == 'pending'
      msg = _("Migration failed, You will want to investigate: https://#{Socket.gethostname}/foreman_tasks/tasks/#{task.id}\n")
      \$stderr.print(msg)
      puts
      fail ForemanTasks::TaskError, task
    else
      puts
      Katello::Pulp3::Migration::CORRUPTABLE_CONTENT_TYPES.each do |type|
        if type.missing_migrated_content.any?
          puts "Some corrupted or missing content found, run 'foreman-maintain content migration-stats' for more information."
          exit(-1)
        end
      end
      puts _("Content Migration completed successfully")
    end
  else
    puts "Migration started, you may monitor it at: https://#{Socket.gethostname}/foreman_tasks/tasks/#{task.id}"
  end
end
EORAKE

Note:  <external_id> must be replaced by the task ID, which is accessible during the content migration, but which is inaccessible during the upgrade (because maintenance mode is automatically enabled during the ugprade process).

The output of the script looks like this:

2022-09-27 18:46:35 -0400: Migrating rpm content to Pulp 3 188999/450904
2022-09-27 18:46:45 -0400: Migrating rpm content to Pulp 3 189079/450904
2022-09-27 18:46:55 -0400: Migrating rpm content to Pulp 3 189079/450904
2022-09-27 18:47:05 -0400: Migrating rpm content to Pulp 3 189303/450904


It would be ideal if these outputs were appended to a log file in /var/log/foreman-maintain so that we could see the progress of each migration/upgrade attempt.


6. For each functional requirement listed, specify how Red Hat and the customer can test to confirm the requirement is successfully implemented.

- install satellite 6.9
- synchronize a couple of RHEL repositories
- upgrade to satellite 6.10
- tail the log
- run an sosreport
- extract the sosreport to verify that the file is contained therein

7. Is there already an existing RFE upstream or in Red Hat Bugzilla?

no

8. Does the customer have any specific time-line dependencies and which release would they like to target (i.e. RHEL5, RHEL6)?

no

9. Is the sales team involved in this request and do they have any additional input?

no

10. List any affected packages or components.

satellite-maintain

11. Would the customer be able to assist in testing this functionality if implemented?

yes