Bug 1993678

Summary: satellite-maintain content migration-stats shows migration time when everything is migrated
Product: Red Hat Satellite Reporter: Kenny Tordeurs <ktordeur>
Component: RepositoriesAssignee: James Jeffers <jjeffers>
Status: CLOSED ERRATA QA Contact: Cole Higgins <chiggins>
Severity: low Docs Contact:
Priority: low    
Version: 6.9.0CC: ahumbe, apatel, aupadhye, bbuckingham, jjeffers, kgaikwad, osousa, peter.vreman
Target Milestone: 6.9.7Keywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: tfm-rubygem-katello-3.18.1.43-1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-11-10 16:23:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Kenny Tordeurs 2021-08-15 11:53:16 UTC
Description of problem:
All content is migrated but migration-stats shows migration time left while it should state that all content is migrated.

Version-Release number of selected component (if applicable):
rubygem-foreman_maintain-0.7.11-1.el7sat.noarch
6.10

How reproducible:
100%

Steps to Reproduce:
1. satellite-maintain content prepare 
2. satellite-maintain content migration-stats
3.

Actual results:
# satellite-maintain content migration-stats
~~~
Running Retrieve Pulp 2 to Pulp 3 migration statistics
================================================================================
Retrieve Pulp 2 to Pulp 3 migration statistics: 
============Migration Summary================
Migrated/Total RPMs: 101121/101121
Migrated/Total errata: 80744/80744
Migrated/Total repositories: 77/77

Estimated migration time based on yum content: fewer than 5 minutes

Note: ensure there is sufficient storage space for /var/lib/pulp/published to double in size before starting the migration process.
Check the size of /var/lib/pulp/published with 'du -sh /var/lib/pulp/published/'
                                                                      [OK]
--------------------------------------------------------------------------------
~~~


Expected results:
Estimated migration time based on yum content: all content has been migrated

Additional info:

Comment 1 Kenny Tordeurs 2021-08-16 07:06:30 UTC
This is actually still on Satellite 6.9.4 while prepping the pulp2>pulp3 for upgrading to 6.10

Comment 2 James Jeffers 2021-08-18 21:35:58 UTC
Created redmine issue https://projects.theforeman.org/issues/33304 from this bug

Comment 4 Bryan Kearney 2021-08-25 20:05:34 UTC
Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/33304 has been resolved.

Comment 6 Kenny Tordeurs 2021-10-18 08:25:49 UTC
Unfortunately the fix would not work for all scenario's.

Here is a sample of a migration that is complete but actually tells us there are 2h and 30min remaining meaning it would not fall under the newly https://github.com/Katello/katello/pull/9565/files#diff-7074496372d9614908d7bc00a42180ccf8de2720f073081247da4ac110c30eccR35 but rather display 95b867a#diff-7074496372d9614908d7bc00a42180ccf8de2720f073081247da4ac110c30eccR33

~~~
# satellite-maintain content migration-stats
Running Retrieve Pulp 2 to Pulp 3 migration statistics
================================================================================
Retrieve Pulp 2 to Pulp 3 migration statistics:
API controllers newer than Apipie cache! Run apipie:cache rake task to regenerate cache.
============Migration Summary================
Migrated/Total RPMs: 156052/156052
Migrated/Total errata: 723178/723178
Migrated/Total repositories: 1486/1486

Estimated migration time based on yum content: 2 hours, 30 minutes

Note: ensure there is sufficient storage space for /var/lib/pulp/published to double in size before starting the migration process.
Check the size of /var/lib/pulp/published with 'du -sh /var/lib/pulp/published/'
                                                                     [OK]
~~~

Either there is some problem with the estimated time remaining or there the new elsif would not cover all scenario's.

Comment 7 Peter Vreman 2021-10-18 11:41:01 UTC
The root cause is the line https://github.com/Katello/katello/blob/KATELLO-3.18/lib/katello/tasks/pulp3_migration_stats.rake#L23

~~~
    migratable_repo_count = ::Katello::Repository.count - ::Katello::Repository.puppet_type.count -
      ::Katello::Repository.ostree_type.count - ::Katello::Repository.deb_type.count
...
 migration_minutes = (0.000646 * on_demand_unmigrated_rpm_count - 3.22 +
                         0.000943 * immediate_unmigrated_rpm_count - 3 +
                         0.0746 * migratable_repo_count).to_i
~~~

The value 'migratable_repo_count' is a static value independent of the migration status and results aways in a value of migration_minutes.

Then later the it-then-else at https://github.com/jjeffers/katello/blob/95b867aa51d263c9d84252ff0ab794daf4ac8ec3/lib/katello/tasks/pulp3_migration_stats.rake#L33 does a wrong order of comparison, instead of first comparing if the migration is completed.

The correct code is to first compare if completed and if not then calculate the time left:

~~~

    puts "============Migration Summary================"
    puts "Migrated/Total RPMs: #{migrated_rpm_count}/#{::Katello::Rpm.count}"
    puts "Migrated/Total errata: #{migrated_erratum_count}/#{::Katello::RepositoryErratum.count}"
    puts "Migrated/Total repositories: #{migrated_repo_count}/#{migratable_repo_count}"

    if migrated_rpm_count == ::Katello::Rpm.count &&
      migrated_erratum_count == ::Katello::RepositoryErratum.count &&
      migrated_repo_count == migratable_repo_count
      puts "All content has been migrated."
    else
      # On Demand RPMs: (6.46E-04)*(#RPMs) + -3.22
      # Immediate RPMs: (9.39E-04)*(#RPMs) + -3
      # Repositories: 0.0746*(#Repos) + -2.07
      migration_minutes = (0.000646 * on_demand_unmigrated_rpm_count - 3.22 +
                           0.000943 * immediate_unmigrated_rpm_count - 3 +
                           0.0746 * migratable_repo_count).to_i
      hours = (migration_minutes / 60) % 60
      minutes = migration_minutes % 60

      # The timing formulas go negative if the amount of content is negligibly small
      if migration_minutes >= 5
        puts "Estimated migration time based on yum content: #{hours} hours, #{minutes} minutes"
	  else
        puts "Estimated migration time based on yum content: fewer than 5 minutes"
      end
    end

~~~

Comment 13 errata-xmlrpc 2021-11-10 16:23:39 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Satellite 6.9.7 Async Bug Fix Update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:4612

Comment 14 Brad Buckingham 2022-12-09 18:25:07 UTC
Clearing needinfo as the bugzilla was closed with 6.9.7.