Bug 1576002

Summary: Satellite 5.8 Fails During schema upgrade to 5.8.0.42
Product: Red Hat Satellite 5 Reporter: Diogo Henrique <dnunes>
Component: UpgradesAssignee: Tomáš Kašpárek <tkasparek>
Status: CLOSED ERRATA QA Contact: Ales Dujicek <adujicek>
Severity: high Docs Contact:
Priority: unspecified    
Version: 580CC: adujicek, akaiser, eherget, ktordeur, rdrazny, tlestach, wpinheir
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: satellite-schema-5.8.0.47-1-sat Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-07-11 15:48:58 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1450111    
Attachments:
Description Flags
schema-upgrade-to-5.8.0.42.patch none

Description Diogo Henrique 2018-05-08 14:10:49 UTC
Description of problem:

When doing the update to schema version 5.8.0.42, schema upgrade breaks with error message: 

UPDATE 0
COMMIT
                                                ?column?                                                 
---------------------------------------------------------------------------------------------------------
 satellite-schema-5.8.0.38-to-satellite-schema-5.8.0.39/001-rhnpackageevr-rhn_pe_v_r_e_uq.sql.postgresql
(1 row)


Version-Release number of selected component (if applicable): 5.8.0.42


How reproducible:

Run a spacewalk-schema-upgrade on Satellite 5.8.0.34-1 to upgrade to 5.8.0.42-1


Steps to Reproduce:
1. Ensure the schema version is 5.8.0.34-1
2. Ensure the packages are updated and run spacewalk-schema-upgrade
3. The schema upgrade process will stop and break

Actual results:

Schema upgrade breaking


Expected results:

Schema upgrade finish without any problem

Additional info:

We believe the upgrade is hanging at this loop:

     for nevra_dup in select id
                from rhnpackagenevra
               where evr_id = duplicate.id loop
        delete from rhnsnapshotpackage where nevra_id = nevra_dup.id and snapshot_id in (select snapshot_id from rhnsnapshotpackage where nevra_id = nevra_orig.id intersect select snapshot_id from rhnsnapshotpackage where nevra_id = nevra_dup.id);
        update rhnsnapshotpackage set nevra_id = nevra_orig.id where nevra_id = nevra_dup.id;  <-==== HGERE
      end loop;


 satellite-schema-5.8.0.38-to-satellite-schema-5.8.0.39/001-rhnpackageevr-rhn_pe_v_r_e_uq.sql.postgresql
(1 row):

CREATE FUNCTION
DROP TRIGGER
psql:/var/log/spacewalk/schema-upgrade/20180503-210515-script.sql:57: ERROR:  null value in column "nevra_id" violates not-null constraint
DETAIL:  Failing row contains (938173, null).
CONTEXT:  SQL statement "update rhnsnapshotpackage set nevra_id = nevra_orig.id where nevra_id = nevra_dup.id"
PL/pgSQL function fix_rhn_pe_v_r_e_uq() line 28 at SQL statement

A possible test at database level, can be:

	update rhnsnapshotpackage set nevra_id = nevra_orig.id where nevra_id = nevra_dup.id and nevra_orig.id is not NULL;

   The key point is to find why that value is coming NULL (if is expected or not) and then make the PL/SQL to handle it correctly. 

We do have two reproducers, one for before the upgrade and one after:

MAY5
-----
petroleomayfive.usersys.redhat.com  --> 10.13.153.214
sys login: root/redhat
web login: satadmin/redhat

MAY3
-----
petroleomaythree.usersys.redhat.com  --> 10.13.153.30
sys login: root/redhat
web login: satadmin/redhat

MAY 3 - Before Schema Upgrade
MAY 5 - After the Schema Upgrade

Case 02091975 Has all history and information of job done trought this environment

Comment 4 Tomáš Kašpárek 2018-05-09 12:49:44 UTC
Created attachment 1433807 [details]
schema-upgrade-to-5.8.0.42.patch

Comment 7 Tomáš Kašpárek 2018-05-23 09:35:13 UTC
spacewalk.git(master): 4898dc4cfc0d3af7488c60ed93f0d7b4da32dcb0

Comment 12 errata-xmlrpc 2018-07-11 15:48:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:2174