Bug 2108719
| Summary: | Upgrading to Satellite 6.11 fails on db:migrate stage with error "null value in column "created_at" violates not-null constraint" | |||
|---|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Sayan Das <saydas> | |
| Component: | Organizations and Locations | Assignee: | satellite6-bugs <satellite6-bugs> | |
| Status: | CLOSED ERRATA | QA Contact: | Shweta Singh <shwsingh> | |
| Severity: | high | Docs Contact: | ||
| Priority: | high | |||
| Version: | 6.11.0 | CC: | ajambhul, ehelms, lpramuk, osousa, pcreech, rlavi, zhunting | |
| Target Milestone: | 6.12.0 | Keywords: | Triaged, Upgrades | |
| Target Release: | Unused | |||
| Hardware: | All | |||
| OS: | Linux | |||
| Whiteboard: | ||||
| Fixed In Version: | foreman-3.3.0.2-1,rubygem-foreman_puppet-4.0.3 | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 2112393 (view as bug list) | Environment: | ||
| Last Closed: | 2022-11-16 13:34:39 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
I don't know how it can even happen as Sat 6.10 also does not have the bad schema but I would really expect that installer will be able to handle this issue in a much better way. I was upgrading from 6.11.1.1 to 6.12.0 (due to another BZ 2120632) and didn't hit this bug. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: Satellite 6.12 Release), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:8506 |
Description of problem: Upgrading to Satellite 6.11 fails on db:migrate stage with error "null value in column "created_at" violates not-null constraint" Version-Release number of selected component (if applicable): Satellite 6.11 [ RHEL 7 ] How reproducible: In customer environments Steps to Reproduce: 1. Install Satellite 6.10 2. Upgrade to 6.11 as per documentation 3. Actual results: Upgrade fails: ~~ \ Executing installer2022-07-19 10:48:00 [ERROR ] [configure] '/usr/sbin/foreman-rake db:migrate' returned 1 instead of one of [0] 2022-07-19 10:48:00 [ERROR ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: change from 'notrun' to ['0'] failed: '/usr/sbin/foreman-rake db:migrate' returned 1 instead of one of [0] | Executing installer2022-07-19 10:48:42 [ERROR ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]: Failed to call refresh: '/usr/sbin/foreman-rake db:migrate' returned 1 instead of one of [0] 2022-07-19 10:48:42 [ERROR ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]: '/usr/sbin/foreman-rake db:migrate' returned 1 instead of one of [0] ~~ Error from logs: ~~ 2022-07-19 10:48:00 [INFO ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: 2022-07-19 10:48:00 [INFO ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: Caused by: 2022-07-19 10:48:00 [INFO ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: PG::NotNullViolation: ERROR: null value in column "created_at" violates not-null constraint 2022-07-19 10:48:00 [INFO ] [configure] /Stage[main]/Foreman::Database/Foreman::Rake[db:migrate]/Exec[foreman-rake-db:migrate]/returns: DETAIL: Failing row contains (321220, 4, 1, ForemanPuppet::Environment, null, null). ~~ "foreman-rake db:migrate" output: ~~~ == 20220208135305 MigrateEnvironmentIgnoreType: migrating ===================== rake aborted! StandardError: An error has occurred, this and all later migrations canceled: PG::NotNullViolation: ERROR: null value in column "created_at" violates not-null constraint DETAIL: Failing row contains (321227, 4, 1, ForemanPuppet::Environment, null, null). ~~~ Expected results: The upgrade should be successful. Additional info: The issue happens as The schema of taxable_taxonomies table somehow is not correct. For a satellite having this issue: ~~~ [root@rhsat ~]$ echo "\d+ taxable_taxonomies" | su - postgres -c "psql foreman" Table "public.taxable_taxonomies" Column | Type | Collation | Nullable | Default | Storage | Stats target | Description --------------+-----------------------------+-----------+----------+------------------------------------------------+----------+--------------+------------- id | integer | | not null | nextval('taxable_taxonomies_id_seq'::regclass) | plain | | taxonomy_id | integer | | | | plain | | taxable_id | integer | | | | plain | | taxable_type | character varying(255) | | | | extended | | created_at | timestamp without time zone | | not null | | plain | | updated_at | timestamp without time zone | | not null | | plain | | Indexes: "taxable_taxonomies_pkey" PRIMARY KEY, btree (id) "taxable_index" UNIQUE, btree (taxable_type, taxable_id, taxonomy_id) "index_taxable_taxonomies_on_taxonomy_id_and_taxable_type" btree (taxonomy_id, taxable_type) Foreign-key constraints: "taxable_taxonomies_taxonomy_id_fk" FOREIGN KEY (taxonomy_id) REFERENCES taxonomies(id) Access method: heap [root@rhsat ~]$ echo "select * from taxable_taxonomies where created_at is null;" | su - postgres -c "psql foreman" id | taxonomy_id | taxable_id | taxable_type | created_at | updated_at ----+-------------+------------+--------------+------------+------------ (0 rows) ~~~ Expected: ~~~ # echo "\d+ taxable_taxonomies" | su - postgres -c "psql foreman" Table "public.taxable_taxonomies" Column | Type | Collation | Nullable | Default | Storage | Stats target | Description --------------+-----------------------------+-----------+----------+------------------------------------------------+----------+--------------+------------- id | integer | | not null | nextval('taxable_taxonomies_id_seq'::regclass) | plain | | taxonomy_id | integer | | | | plain | | taxable_id | integer | | | | plain | | taxable_type | character varying(255) | | | | extended | | created_at | timestamp without time zone | | | | plain | | updated_at | timestamp without time zone | | | | plain | | Indexes: "taxable_taxonomies_pkey" PRIMARY KEY, btree (id) "taxable_index" UNIQUE, btree (taxable_type, taxable_id, taxonomy_id) "index_taxable_taxonomies_on_taxonomy_id_and_taxable_type" btree (taxonomy_id, taxable_type) Foreign-key constraints: "taxable_taxonomies_taxonomy_id_fk" FOREIGN KEY (taxonomy_id) REFERENCES taxonomies(id) Access method: heap ~~~ Workaround: # su - postgres -c "psql foreman" ALTER TABLE taxable_taxonomies ALTER COLUMN created_at DROP NOT NULL; ALTER TABLE taxable_taxonomies ALTER COLUMN updated_at DROP NOT NULL; exit # foreman-rake db:migrate --trace If no issues are reported this time, proceed with re-running the upgrade.