Bug 2218878

Summary: Upgrade to Satellite 6.12 can fail if some on_demand repositories have checksum_type set
Product: Red Hat Satellite Reporter: Sayan Das <saydas>
Component: UpgradesAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED ERRATA QA Contact: Sam Bible <sbible>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.12.4CC: jpathan, osousa, pcreech, rlavi, sadas, sajha, sbible
Target Milestone: 6.14.0Keywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: rubygem-katello-4.9.0.3-1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2227898 2231498 (view as bug list) Environment:
Last Closed: 2023-11-08 14:19:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sayan Das 2023-06-30 11:43:45 UTC
Description of problem:

If a sat 6.11 has a couple of repositories enabled with the on_demand download policy and some of them have checksum_type set, Then upgrading to 6.12 will fail on the db migrate step.


Version-Release number of selected component (if applicable):

Satellite 6.12 ( being upgraded from satellite 6.11 )

How reproducible:

Under specific circumstances


Steps to Reproduce:
1. Install satellite 6.11 and have repos enabled like this

# echo "select id,name,download_policy,checksum_type from katello_root_repositories;" | su - postgres -c "psql foreman"
 id |                           name                           | download_policy | checksum_type 
----+----------------------------------------------------------+-----------------+---------------
  1 | Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server    | on_demand       | 
  2 | Red Hat Enterprise Linux 8 for x86_64 - BaseOS RPMs 8    | on_demand       | 
  3 | Red Hat Enterprise Linux 8 for x86_64 - AppStream RPMs 8 | on_demand       | 
  4 | Red Hat Enterprise Linux 9 for x86_64 - BaseOS RPMs 9    | on_demand       | 
  5 | Red Hat Enterprise Linux 9 for x86_64 - AppStream RPMs 9 | on_demand       | 
  8 | Red Hat Satellite Client 6 for RHEL 9 x86_64 RPMs        | on_demand       | 
  6 | Red Hat Satellite Client 6 for RHEL 7 Server RPMs x86_64 | on_demand       | 
  7 | Red Hat Satellite Client 6 for RHEL 8 x86_64 RPMs        | on_demand       | 

2. Set checksum_type for some repos:

# echo "update katello_root_repositories set checksum_type = 'sha256' where content_type='yum' and name like '%Satellite Client 6%';" | su - postgres -c "psql foreman"
UPDATE 3


3. Try upgrading to Satellite 6.12


Actual results:

Upgrade fails here on db:migrate step:

2023-06-30 13:31:08 [NOTICE] [configure] 500 configuration steps out of 1529 steps complete.
2023-06-30 13:31:11 [NOTICE] [configure] 750 configuration steps out of 1533 steps complete.
2023-06-30 13:31:14 [NOTICE] [configure] 1000 configuration steps out of 1575 steps complete.
2023-06-30 13:31:15 [NOTICE] [configure] 1250 configuration steps out of 1575 steps complete.
2023-06-30 13:32:07 [ERROR ] [configure] '/usr/sbin/foreman-rake db:migrate' returned 1 instead of one of [0]
2023-06-30 13:32:07 [ERROR ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: change from 'notrun' to ['0'] failed: '/usr/sbin/foreman-rake db:migrate' returned 1 instead of one of [0]
2023-06-30 13:32:42 [ERROR ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]: Failed to call refresh: '/usr/sbin/foreman-rake db:migrate' returned 1 instead of one of [0]
2023-06-30 13:32:42 [ERROR ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]: '/usr/sbin/foreman-rake db:migrate' returned 1 instead of one of [0]
2023-06-30 13:33:20 [NOTICE] [configure] 1500 configuration steps out of 1575 steps complete.
2023-06-30 13:33:37 [NOTICE] [configure] System configuration has finished.



db:migrate fails to apply the RemoveDrpmFromIgnorableContent migration 


# foreman-rake db:migrate --trace
** Invoke db:migrate (first_time)
** Invoke db:load_config (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute db:load_config
** Invoke plugin:refresh_migrations (first_time)
** Invoke environment 
** Execute plugin:refresh_migrations
** Execute db:migrate
== 20220228173251 RemoveDrpmFromIgnorableContent: migrating ===================
rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:

Validation failed: Checksum type Checksum type cannot be set for yum repositories with on demand download policy.
/usr/share/gems/gems/activerecord-6.0.6/lib/active_record/validations.rb:80:in `raise_validation_error'
/usr/share/gems/gems/activerecord-6.0.6/lib/active_record/validations.rb:53:in `save!'
/usr/share/gems/gems/activerecord-6.0.6/lib/active_record/transactions.rb:318:in `block in save!'
/usr/share/gems/gems/activerecord-6.0.6/lib/active_record/transactions.rb:375:in `block in with_transaction_returning_status'
/usr/share/gems/gems/activerecord-6.0.6/lib/active_record/connection_adapters/abstract/database_statements.rb:278:in `transaction'
/usr/share/gems/gems/activerecord-6.0.6/lib/active_record/transactions.rb:212:in `transaction'
/usr/share/gems/gems/activerecord-6.0.6/lib/active_record/transactions.rb:366:in `with_transaction_returning_status'
/usr/share/gems/gems/activerecord-6.0.6/lib/active_record/transactions.rb:318:in `save!'
/usr/share/gems/gems/activerecord-6.0.6/lib/active_record/suppressor.rb:48:in `save!'
/usr/share/gems/gems/katello-4.5.0.33/db/migrate/20220228173251_remove_drpm_from_ignorable_content.rb:9:in `block in up'
/usr/share/gems/gems/katello-4.5.0.33/db/migrate/20220228173251_remove_drpm_from_ignorable_content.rb:3:in `each'
/usr/share/gems/gems/katello-4.5.0.33/db/migrate/20220228173251_remove_drpm_from_ignorable_content.rb:3:in `up'


Expected results:

No such errors.


Additional info:


The issue happens when repos with on_Demand download policy have checksum_type set and root.save being executed as a part of the migration https://github.com/Katello/katello/blob/master/db/migrate/20220228173251_remove_drpm_from_ignorable_content.rb#L9 


The fix\workaround is simple i.e. 

* Apply this patch

# git diff
diff --git a/db/migrate/20220228173251_remove_drpm_from_ignorable_content.rb b/db/migrate/20220228173251_remove_drpm_from_ignorable_content.rb
index 1ff0eb9..a0c6320 100644
--- a/db/migrate/20220228173251_remove_drpm_from_ignorable_content.rb
+++ b/db/migrate/20220228173251_remove_drpm_from_ignorable_content.rb
@@ -6,6 +6,7 @@ class RemoveDrpmFromIgnorableContent < ActiveRecord::Migration[6.0]
       else
         root.ignorable_content = []
       end
+      root.checksum_type = nil if root.download_policy == ::Katello::RootRepository::DOWNLOAD_ON_DEMAND
       root.save!
     end
   end

* Re-run the db:migrate and then retry the upgrade.

Comment 2 Sayan Das 2023-06-30 19:27:19 UTC
Connecting the redmine

Bug #36562: Upgrade to Katello 4.5 can fail if some on_demand repositories have checksum_type set - Katello - Foreman
https://projects.theforeman.org/issues/36562

Comment 3 Bryan Kearney 2023-07-11 16:03:17 UTC
Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/36562 has been resolved.

Comment 5 Sam Bible 2023-08-03 15:41:50 UTC
Verified on 6.14 Snap 10

After discussion with Dev, this BZ doesn't have any additional steps to verify other than dbmigration continuing to work in 6.14, which it does, so this will be considered verified for 6.14.

Comment 8 errata-xmlrpc 2023-11-08 14:19:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.14 security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:6818