Bug 1344456 - replication errors after inplace upgrade from 5.5 to 5.6.0.10
Summary: replication errors after inplace upgrade from 5.5 to 5.6.0.10
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: Appliance
Version: 5.6.0
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: GA
: 5.6.0
Assignee: Nick Carboni
QA Contact: luke couzens
URL:
Whiteboard: upgrade:black
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-06-09 18:06 UTC by luke couzens
Modified: 2016-06-29 16:07 UTC (History)
6 users (show)

Fixed In Version: 5.6.0.12
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-06-29 16:07:22 UTC
Category: ---
Cloudforms Team: ---
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:1348 0 normal SHIPPED_LIVE CFME 5.6.0 bug fixes and enhancement update 2016-06-29 18:50:04 UTC

Description luke couzens 2016-06-09 18:06:46 UTC
Description of problem:replication errors after inplace upgrade from 5.5 to 5.6.0.10


Version-Release number of selected component (if applicable):5.6.0.10


How reproducible:100%


Steps to Reproduce:
1.2 appliance's in replicated setup
2.add update.repo with latest 5.6 repos to /etc/yum.repos.d/
3.stop replication sync in ui
4.run systemctl stop evmserverd 
5.run yum update following docs

Actual results:replication shows as active but backlog increases and errors show in logs.


Expected results: replication works correctly


Additional info:
[0]
Documentation https://access.redhat.com/articles/2297391
evm.log http://pastebin.test.redhat.com/382304

apps ips
rro 10.16.5.145
rr99 10.8.58.217

Comment 2 Nick Carboni 2016-06-16 21:07:12 UTC
https://github.com/ManageIQ/manageiq/pull/9264

Comment 3 CFME Bot 2016-06-20 18:36:05 UTC
New commit detected on ManageIQ/manageiq/master:
https://github.com/ManageIQ/manageiq/commit/b1cb72ee2c059864ff2fc6b7ee5f295153c63a83

commit b1cb72ee2c059864ff2fc6b7ee5f295153c63a83
Author:     Nick Carboni <ncarboni>
AuthorDate: Thu Jun 16 16:54:24 2016 -0400
Commit:     Nick Carboni <ncarboni>
CommitDate: Mon Jun 20 11:08:36 2016 -0400

    Fix replication installation after primary key addition
    
    In 7656212 a migration added "id" primary keys to some
    tables that were being replicated. The triggers used in
    rubyrep replication hardcode the "key" used to retrieve
    the row that has been changed. In tables without a proper PK
    these triggers were using multiple columns for this key.
    
    Before this change, a migrated environment would have the same (old)
    triggers firing on tables that now had a primary key which would
    write the old key into the pending changes table. This mismatch
    between what was recorded in the pending changes table and the
    actual primary key caused an infinate loop / timeout during
    rubyrep replication.
    
    This fixes the situation by removing the triggers and other
    replication state information for these tables and allowing
    them to be rebuilt when replication starts again.
    
    The new triggers will have the new primary key in them and
    will generate changes that can be properly processed.
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1344456

 ..._rep_triggers_on_tables_with_new_primary_key.rb | 34 ++++++++++++++++++++++
 ...triggers_on_tables_with_new_primary_key_spec.rb | 27 +++++++++++++++++
 2 files changed, 61 insertions(+)
 create mode 100644 db/migrate/20160425161345_reset_ruby_rep_triggers_on_tables_with_new_primary_key.rb
 create mode 100644 spec/migrations/20160425161345_reset_ruby_rep_triggers_on_tables_with_new_primary_key_spec.rb

Comment 4 luke couzens 2016-06-23 15:00:55 UTC
Verified in 5.6.0.12

Comment 6 errata-xmlrpc 2016-06-29 16:07:22 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1348


Note You need to log in before you can comment on or make changes to this bug.