Description of problem: Satellite upgrade fails during db:migrate with PG::ForeignKeyViolation: ERROR: update or delete on table "katello_errata" violates foreign key constraint "katello_content_facet_errata_errata_id" on table "katello_content_facet_errata" QE populated Satellite was used. Version-Release number of selected component (if applicable): 6.11.0 Snap16 How reproducible: deterministic Steps to Reproduce: 1. Have a QE populated and registered Satellite 6.10 2. Upgrade to 6.11.0 # satellite-maintain upgrade run --target-version 6.11 -w repositories-validate,repositories-setup -y ... Running Migration scripts to Satellite 6.11 ================================================================================ Setup repositories: [SKIPPED] -------------------------------------------------------------------------------- Unlock packages: [OK] -------------------------------------------------------------------------------- Update package(s) : [OK] -------------------------------------------------------------------------------- Procedures::Installer::Upgrade: [FAIL] Failed executing LANG=en_US.utf-8 satellite-installer --disable-system-checks, exit status 6: 2022-04-14 07:12:06 [NOTICE] [root] Loading installer configuration. This will take some time. 2022-04-14 07:12:13 [NOTICE] [root] Running installer with log based terminal output at level NOTICE. 2022-04-14 07:12:13 [NOTICE] [root] Use -l to set the terminal output log level to ERROR, WARN, NOTICE, INFO, or DEBUG. See --full-help for definitions. 2022-04-14 07:12:20 [WARN ] [pre] Skipping system checks. 2022-04-14 07:12:20 [WARN ] [pre] Skipping system checks. 2022-04-14 07:12:33 [NOTICE] [configure] Starting system configuration. 2022-04-14 07:12:49 [NOTICE] [configure] 250 configuration steps out of 2165 steps complete. 2022-04-14 07:13:02 [NOTICE] [configure] 500 configuration steps out of 2169 steps complete. 2022-04-14 07:13:12 [NOTICE] [configure] 750 configuration steps out of 2171 steps complete. 2022-04-14 07:13:23 [NOTICE] [configure] 1000 configuration steps out of 2176 steps complete. 2022-04-14 07:13:38 [NOTICE] [configure] 1250 configuration steps out of 2194 steps complete. 2022-04-14 07:14:58 [ERROR ] [configure] '/usr/sbin/foreman-rake db:migrate' returned 1 instead of one of [0] 2022-04-14 07:14:58 [ERROR ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: change from 'notrun' to ['0'] failed: '/usr/sbin/foreman-rake db:migrate' returned 1 instead of one of [0] 2022-04-14 07:16:19 [ERROR ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]: Failed to call refresh: '/usr/sbin/foreman-rake db:migrate' returned 1 instead of one of [0] 2022-04-14 07:16:19 [ERROR ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]: '/usr/sbin/foreman-rake db:migrate' returned 1 instead of one of [0] 2022-04-14 07:16:49 [NOTICE] [configure] 1500 configuration steps out of 2195 steps complete. 2022-04-14 07:16:53 [NOTICE] [configure] 1750 configuration steps out of 3039 steps complete. 2022-04-14 07:16:54 [NOTICE] [configure] 2000 configuration steps out of 3039 steps complete. 2022-04-14 07:16:54 [NOTICE] [configure] 2250 configuration steps out of 3039 steps complete. 2022-04-14 07:16:55 [NOTICE] [configure] 2500 configuration steps out of 3039 steps complete. 2022-04-14 07:17:15 [NOTICE] [configure] 2750 configuration steps out of 3039 steps complete. 2022-04-14 07:17:52 [NOTICE] [configure] 3000 configuration steps out of 3039 steps complete. 2022-04-14 07:18:10 [NOTICE] [configure] System configuration has finished. ... Actual results: failed upgrade due to failing db:migrate Expected results: successful upgrade
# foreman-rake db:migrate 'ErbParser' is ignored. 'RubyParser' is ignored. API controllers newer than Apipie cache! Run apipie:cache rake task to regenerate cache. == 20220303160220 RemoveDuplicateErrata: migrating ============================ rake aborted! StandardError: An error has occurred, this and all later migrations canceled: PG::ForeignKeyViolation: ERROR: update or delete on table "katello_errata" violates foreign key constraint "katello_content_facet_errata_errata_id" on table "katello_content_facet_errata" DETAIL: Key (id)=(12675) is still referenced from table "katello_content_facet_errata". /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/postgresql_adapter.rb:675:in `async_exec_params' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/postgresql_adapter.rb:675:in `block (2 levels) in exec_no_cache' /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads' /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares' /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/postgresql_adapter.rb:674:in `block in exec_no_cache' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/abstract_adapter.rb:722:in `block (2 levels) in log' /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize' /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt' /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize' /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt' /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/abstract_adapter.rb:721:in `block in log' /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/notifications/instrumenter.rb:24:in `instrument' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/abstract_adapter.rb:712:in `log' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/postgresql_adapter.rb:673:in `exec_no_cache' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/postgresql_adapter.rb:656:in `execute_and_clear' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/postgresql/database_statements.rb:111:in `exec_delete' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/abstract/database_statements.rb:180:in `delete' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/abstract/query_cache.rb:22:in `delete' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/relation.rb:587:in `delete_all' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.3.0.20/db/migrate/20220303160220_remove_duplicate_errata.rb:32:in `block in up' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.3.0.20/db/migrate/20220303160220_remove_duplicate_errata.rb:16:in `each' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.3.0.20/db/migrate/20220303160220_remove_duplicate_errata.rb:16:in `up' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/migration.rb:831:in `exec_migration' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/migration.rb:812:in `block (2 levels) in migrate' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/migration.rb:811:in `block in migrate' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:471:in `with_connection' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/migration.rb:810:in `migrate' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/migration.rb:1002:in `migrate' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/migration.rb:1310:in `block in execute_migration_in_transaction' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/migration.rb:1361:in `block in ddl_transaction' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/abstract/database_statements.rb:280:in `block in transaction' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/abstract/transaction.rb:280:in `block in within_new_transaction' /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize' /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt' /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize' /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt' /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/abstract/transaction.rb:278:in `within_new_transaction' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/abstract/database_statements.rb:280:in `transaction' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/transactions.rb:212:in `transaction' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/migration.rb:1361:in `ddl_transaction' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/migration.rb:1309:in `execute_migration_in_transaction' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/migration.rb:1281:in `block in migrate_without_lock' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/migration.rb:1280:in `each' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/migration.rb:1280:in `migrate_without_lock' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/migration.rb:1229:in `block in migrate' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/migration.rb:1382:in `with_advisory_lock' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/migration.rb:1229:in `migrate' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/migration.rb:1061:in `up' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/migration.rb:1036:in `migrate' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/tasks/database_tasks.rb:238:in `migrate' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/railties/databases.rake:86:in `block (3 levels) in <top (required)>' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/railties/databases.rake:84:in `each' /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/railties/databases.rake:84:in `block (2 levels) in <top (required)>' /opt/rh/rh-ruby27/root/usr/share/gems/gems/rake-13.0.1/exe/rake:27:in `<top (required)>' ...
Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/34771 has been resolved.
Verified. Upgraded to 6.11.0-19.0 # rpm -qa | grep rubygem-katello tfm-rubygem-katello-4.3.0.33-1.el7sat.noarch Upgrade successful. Thanks sajha for helping in verifying the BZ.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: Satellite 6.11 Release), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:5498