Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1993844 - The system appears to already be using pulp3 with all content migrated.
Summary: The system appears to already be using pulp3 with all content migrated.
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Satellite Maintain
Version: 6.9.0
Hardware: x86_64
OS: Linux
high
high
Target Milestone: 6.9.6
Assignee: Justin Sherrill
QA Contact: Gaurav Talreja
URL:
Whiteboard:
: 1995694 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-08-16 09:19 UTC by Kenny Tordeurs
Modified: 2023-09-15 01:13 UTC (History)
10 users (show)

Fixed In Version: tfm-rubygem-katello-3.18.1.40-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-09-21 14:37:03 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 32831 0 None None None 2021-08-16 16:33:18 UTC
Red Hat Product Errata RHBA-2021:3627 0 None None None 2021-09-21 14:37:04 UTC

Description Kenny Tordeurs 2021-08-16 09:19:03 UTC 
Description of problem:
Upgrade fails stating the pulp3 content is already migrated.

--------------------------------------------------------------------------------
Switch support for certain content from Pulp 2 to Pulp 3: 
Performing final content migration before switching content           [FAIL]
Failed executing foreman-rake katello:pulp3_migration, exit status 1:
 rake aborted!
The system appears to already be using pulp3 with all content migrated.
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.33/lib/katello/tasks/check_config.rake:15:in `block (2 levels) in <top (required)>'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/exe/rake:27:in `<top (required)>'
Tasks: TOP => katello:pulp3_migration => katello:check_config
(See full trace by running task with --trace)
Checking for valid Katello configuraton.
--------------------------------------------------------------------------------


Version-Release number of selected component (if applicable):
6.9.4
rubygem-foreman_maintain-0.8.10-1.el7sat.noarch

How reproducible:
100%

Steps to Reproduce:
1. All content is migrated before upgrade
2. satellite-maintain upgrade run --target-version 6.10
3. 

Actual results:
--------------------------------------------------------------------------------
Switch support for certain content from Pulp 2 to Pulp 3: 
Performing final content migration before switching content           [FAIL]
Failed executing foreman-rake katello:pulp3_migration, exit status 1:
 rake aborted!
The system appears to already be using pulp3 with all content migrated.
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.33/lib/katello/tasks/check_config.rake:15:in `block (2 levels) in <top (required)>'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/exe/rake:27:in `<top (required)>'
Tasks: TOP => katello:pulp3_migration => katello:check_config
(See full trace by running task with --trace)
Checking for valid Katello configuraton.
--------------------------------------------------------------------------------


Expected results:
Upgrade to continue and be OK for the task as the content is already migrated

Additional info:

# foreman-rake katello:pulp3_migration --trace
** Invoke katello:pulp3_migration (first_time)
** Invoke environment (first_time)
** Execute environment
** Invoke katello:disable_dynflow (first_time)
** Execute katello:disable_dynflow
** Invoke katello:check_config (first_time)
** Invoke environment 
** Execute katello:check_config
Checking for valid Katello configuraton.
rake aborted!
The system appears to already be using pulp3 with all content migrated.
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.33/lib/katello/tasks/check_config.rake:15:in `block (2 levels) in <top (required)>'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/task.rb:273:in `block in execute'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/task.rb:273:in `each'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/task.rb:273:in `execute'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/task.rb:214:in `block in invoke_with_call_chain'
/opt/rh/rh-ruby25/root/usr/share/ruby/monitor.rb:235:in `mon_synchronize'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/task.rb:194:in `invoke_with_call_chain'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/task.rb:238:in `block in invoke_prerequisites'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/task.rb:236:in `each'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/task.rb:236:in `invoke_prerequisites'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/task.rb:213:in `block in invoke_with_call_chain'
/opt/rh/rh-ruby25/root/usr/share/ruby/monitor.rb:235:in `mon_synchronize'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/task.rb:194:in `invoke_with_call_chain'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/task.rb:183:in `invoke'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/application.rb:160:in `invoke_task'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/application.rb:116:in `each'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/application.rb:116:in `block in top_level'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/application.rb:125:in `run_with_threads'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/application.rb:110:in `top_level'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/application.rb:83:in `block in run'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/application.rb:186:in `standard_exception_handling'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/application.rb:80:in `run'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/exe/rake:27:in `<top (required)>'
/opt/rh/rh-ruby25/root/usr/bin/rake:23:in `load'
/opt/rh/rh-ruby25/root/usr/bin/rake:23:in `<main>'
Tasks: TOP => katello:pulp3_migration => katello:check_config

# foreman-rake katello:check_config --trace
** Invoke katello:check_config (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute katello:check_config
Checking for valid Katello configuraton.
rake aborted!
The system appears to already be using pulp3 with all content migrated.
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.33/lib/katello/tasks/check_config.rake:15:in `block (2 levels) in <top (required)>'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/task.rb:273:in `block in execute'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/task.rb:273:in `each'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/task.rb:273:in `execute'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/task.rb:214:in `block in invoke_with_call_chain'
/opt/rh/rh-ruby25/root/usr/share/ruby/monitor.rb:235:in `mon_synchronize'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/task.rb:194:in `invoke_with_call_chain'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/task.rb:183:in `invoke'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/application.rb:160:in `invoke_task'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/application.rb:116:in `each'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/application.rb:116:in `block in top_level'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/application.rb:125:in `run_with_threads'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/application.rb:110:in `top_level'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/application.rb:83:in `block in run'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/application.rb:186:in `standard_exception_handling'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/lib/rake/application.rb:80:in `run'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/exe/rake:27:in `<top (required)>'
/opt/rh/rh-ruby25/root/usr/bin/rake:23:in `load'
/opt/rh/rh-ruby25/root/usr/bin/rake:23:in `<main>'
Tasks: TOP => katello:check_config
Comment 2 Sayan Das 2021-08-16 13:47:50 UTC 
Yeah, having exact same situation here.

I performed pulp2 to 3 migration manually and hit this later during upgrade.

--------------------------------------------------------------------------------
Switch support for certain content from Pulp 2 to Pulp 3: 
Performing final content migration before switching content           [FAIL]
Failed executing foreman-rake katello:pulp3_migration, exit status 1:
 rake aborted!
The system appears to already be using pulp3 with all content migrated.
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.32/lib/katello/tasks/check_config.rake:15:in `block (2 levels) in <top (required)>'
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/exe/rake:27:in `<top (required)>'
Tasks: TOP => katello:pulp3_migration => katello:check_config
(See full trace by running task with --trace)
Checking for valid Katello configuraton.
--------------------------------------------------------------------------------
Scenario [Migration scripts to Satellite 6.10] failed.

The following steps ended up in failing state:

  [content-switchover]

Resolve the failed steps and rerun
the command. In case the failures are false positives,
use --whitelist="content-switchover"





So had to whitelist content-switchover to get past this but then upgrade was successful.
Comment 3 Sayan Das 2021-08-16 13:49:19 UTC 
(In reply to Sayan Das from comment #2)
> Yeah, having exact same situation here.
> 
> I performed pulp2 to 3 migration manually and hit this later during upgrade.
> 
> -----------------------------------------------------------------------------
> ---
> Switch support for certain content from Pulp 2 to Pulp 3: 
> Performing final content migration before switching content           [FAIL]
> Failed executing foreman-rake katello:pulp3_migration, exit status 1:
>  rake aborted!
> The system appears to already be using pulp3 with all content migrated.
> /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.32/lib/katello/
> tasks/check_config.rake:15:in `block (2 levels) in <top (required)>'
> /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.3/exe/rake:27:in `<top
> (required)>'
> Tasks: TOP => katello:pulp3_migration => katello:check_config
> (See full trace by running task with --trace)
> Checking for valid Katello configuraton.
> -----------------------------------------------------------------------------
> ---
> Scenario [Migration scripts to Satellite 6.10] failed.
> 
> The following steps ended up in failing state:
> 
>   [content-switchover]
> 
> Resolve the failed steps and rerun
> the command. In case the failures are false positives,
> use --whitelist="content-switchover"
> 
> 
> 
> 
> 
> So had to whitelist content-switchover to get past this but then upgrade was
> successful.


If foreman-maintain can detect "The system appears to already be using pulp3 with all content migrated" , then it should not mark the step as failed. It should either handle it correctly and mark it as PASS or mark it as WARNING with some proper reasoning.
Comment 4 Justin Sherrill 2021-08-16 16:33:17 UTC 
Connecting redmine issue https://projects.theforeman.org/issues/32831 from this bug
Comment 8 Justin Sherrill 2021-08-19 18:35:34 UTC 
*** Bug 1995694 has been marked as a duplicate of this bug. ***
Comment 13 Gaurav Talreja 2021-09-09 15:15:17 UTC 
Verified.

Tested on Satellite 6.9.6 Snap 1.0
Version: rubygem-foreman_maintain-0.8.13-2.el7sat.noarch

Steps:
1. Perform pulp2to3 migration
2. Run twice # foreman-maintain upgrade run --target-version 6.10 --whitelist="repositories-validate" (Fails post-switchover for repositories-setup)

Observation:
For upgrade reruns(which failed post-switchover), migration/switchover foreman-rake tasks just get skipped during the content-switchover step, like
 
--------------------------------------------------------------------------------
Switch support for certain content from Pulp 2 to Pulp 3:
Performing final content migration before switching content
Checking for valid Katello configuraton.
Switchover is already complete, skipping migration work.
Performing a check to verify everything that is needed has been migrated

Switching specified content over to pulp 3
Checking for valid Katello configuraton.
Switchover is already complete, skipping switchover task.
Re-running the installer to switch specified content over to pulp3    [OK]
--------------------------------------------------------------------------------
Comment 16 errata-xmlrpc 2021-09-21 14:37:03 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Satellite Maintenance 6.9.6 Async Bug Fix Update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:3627
Comment 17 Red Hat Bugzilla 2023-09-15 01:13:43 UTC 
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 500 days
Note You need to log in before you can comment on or make changes to this bug.