Bug 1289156 - Replication: host_storages table, duplicate key
Replication: host_storages table, duplicate key
Status: CLOSED ERRATA
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Appliance (Show other bugs)
5.5.0
Unspecified Unspecified
high Severity high
: GA
: 5.6.0
Assigned To: Nick Carboni
Alex Newman
replication
: ZStream
Depends On:
Blocks: 1289200
  Show dependency treegraph
 
Reported: 2015-12-07 09:43 EST by Jan Krocil
Modified: 2016-06-29 11:18 EDT (History)
10 users (show)

See Also:
Fixed In Version: 5.6.0.0
Doc Type: Bug Fix
Doc Text:
In the previous version of CloudForms Management Engine, replication failed with duplicate key present in the `host_storages` table on master. This was happening because a migration was adding a primary key to the host_storages table and in the process altering existing rows to have a key within the range defined by the current database region. This caused existing rows in the host_storages table to be assigned primary keys in the master database region which resulted in rows with ids from multiple regions (master and remote regions). Replication determines whether a row exists on the master side by comparing primary keys. Thereby, the primary key did not match for a row where it should. This caused replication to attempt to reinsert the row which failed because of a unique index on two other columns in host_storages. This patch adds migration to fix host_storages replication issue. This removes the entries in the host_storages table on the target region and resets the replication triggers on the source regions which has now resolved the issue.
Story Points: ---
Clone Of:
: 1289200 (view as bug list)
Environment:
Last Closed: 2016-06-29 11:18:04 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Article) 2076193 None None None Never

  None (edit)
Comment 2 Nick Carboni 2015-12-07 10:09:50 EST
This is happening because a migration (https://github.com/ManageIQ/manageiq/blob/master/db/migrate/20150921204114_add_vmware_ro_datastores_to_hosts_storages.rb) is adding a primary key to the host_storages table and in the process altering existing rows to have a key within the range defined by the current database region.

This causes existing rows in the host_storages table to be assigned primary keys in the master database region which will result in rows with ids from multiple regions (master and remote regions).

Replication determines whether a row exists on the master side by comparing primary keys. In this case the primary key will not match for a row where it should. This causes replication to attempt to reinsert the row which fails because of a unique index on two other columns in host_storages.
Comment 3 Marianne Feifer 2015-12-07 14:58:00 EST
Added link to migration article
Comment 4 Nick Carboni 2015-12-08 16:44:07 EST
https://github.com/ManageIQ/manageiq/pull/5760
Comment 5 CFME Bot 2015-12-09 10:03:52 EST
New commit detected on ManageIQ/manageiq/master:
https://github.com/ManageIQ/manageiq/commit/04b785eee3771861e3d9a9e7cd78064a721fb0e9

commit 04b785eee3771861e3d9a9e7cd78064a721fb0e9
Author:     Nick Carboni <ncarboni@redhat.com>
AuthorDate: Tue Dec 8 16:06:55 2015 -0500
Commit:     Nick Carboni <ncarboni@redhat.com>
CommitDate: Wed Dec 9 08:52:56 2015 -0500

    Add migration to fix host_storages replication issue
    
    After we run the migration added in commit 04f2fa211b8ebeb0557ca3712af846c9b556869f
    the table host_storages will be assigned a primary key column and existing
    rows will be given a value within the current region's range.
    
    This causes an issue when run on a replication target as the id should
    match the one on the replication source region not the target one.
    
    This fix removes the entries in the host_storages table on the target
    region and resets the replication triggers on the source regions.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1289156

 ...0956_fix_host_storage_replication_on_upgrade.rb | 49 ++++++++++++++++++++++
 ...fix_host_storage_replication_on_upgrade_spec.rb | 47 +++++++++++++++++++++
 2 files changed, 96 insertions(+)
 create mode 100644 db/migrate/20151208150956_fix_host_storage_replication_on_upgrade.rb
 create mode 100644 spec/migrations/20151208150956_fix_host_storage_replication_on_upgrade_spec.rb
Comment 7 errata-xmlrpc 2016-06-29 11:18:04 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1348

Note You need to log in before you can comment on or make changes to this bug.