It is possible to configure different sets of tables as excluded from replication on different servers. This causes each worker which reloads their settings to reevaluate the replication set and possibly add or remove the table depending on that server's specific settings. The exclude tables should be a region level setting and not able to be changed on a server by server basis. This makes errors like https://bugzilla.redhat.com/show_bug.cgi?id=1387420 and https://bugzilla.redhat.com/show_bug.cgi?id=1380475 more likely.
I'm working on a fix for this issue that will move the exclude tables out of the settings in favor of reading the configuration directly from the pglogical tables in the database and using a separate yml file for the default excludes. In order to properly change these settings, we will need a new UI for adding/removing tables from the replication excludes on the region -> replication tab rather than in the advanced settings. This is required because all of the advanced settings are saved as related to the server which is the source of the issue.
https://github.com/ManageIQ/manageiq/pull/12592
New commit detected on ManageIQ/manageiq/master: https://github.com/ManageIQ/manageiq/commit/0b3975ada8433e5fef3cc4d58a230cee4c0b192d commit 0b3975ada8433e5fef3cc4d58a230cee4c0b192d Author: Nick Carboni <ncarboni> AuthorDate: Thu Nov 10 18:07:00 2016 -0500 Commit: Nick Carboni <ncarboni> CommitDate: Fri Nov 11 15:53:35 2016 -0500 Get the default excludes from the new file rather than Settings. This also exposes a new method, `#active_excludes`, to retrieve the current excludes being used by pglogical. This will pull the list of tables directly from the pglogical tables in the database rather than using an intermediate list of settings related to the server. https://bugzilla.redhat.com/show_bug.cgi?id=1389821 lib/miq_pglogical.rb | 24 ++++++++++++++++++------ spec/replication/util/miq_pglogical_spec.rb | 9 +++------ 2 files changed, 21 insertions(+), 12 deletions(-)
New commit detected on ManageIQ/manageiq/master: https://github.com/ManageIQ/manageiq/commit/f50c7986d9d2153eed67af2569c1a35a3ddcad68 commit f50c7986d9d2153eed67af2569c1a35a3ddcad68 Author: Nick Carboni <ncarboni> AuthorDate: Thu Nov 10 18:05:31 2016 -0500 Commit: Nick Carboni <ncarboni> CommitDate: Fri Nov 11 15:53:30 2016 -0500 Move the default replication excludes out of the settings.yml file This will allow us to move away from storing this database centric setting as related to the server. https://bugzilla.redhat.com/show_bug.cgi?id=1389821 config/default_replication_exclude_tables.yml | 92 ++++++++++++++++++++++++++ config/settings.yml | 93 --------------------------- 2 files changed, 92 insertions(+), 93 deletions(-) create mode 100644 config/default_replication_exclude_tables.yml
New commit detected on ManageIQ/manageiq/master: https://github.com/ManageIQ/manageiq/commit/b3969a73926e03e743569bf7d0236943737cfc6c commit b3969a73926e03e743569bf7d0236943737cfc6c Author: Nick Carboni <ncarboni> AuthorDate: Fri Nov 11 11:25:55 2016 -0500 Commit: Nick Carboni <ncarboni> CommitDate: Fri Nov 11 15:53:41 2016 -0500 Add class methods for refreshing the excludes from the queue We can't use the instance methods with the queue because this is not an ActiveRecord object. https://bugzilla.redhat.com/show_bug.cgi?id=1389821 lib/miq_pglogical.rb | 14 ++++++++++++++ spec/replication/util/miq_pglogical_spec.rb | 13 +++++++++++++ 2 files changed, 27 insertions(+)
New commit detected on ManageIQ/manageiq/master: https://github.com/ManageIQ/manageiq/commit/991a3d4d1b7dfbb73fdedfe12b19e576863c8e5c commit 991a3d4d1b7dfbb73fdedfe12b19e576863c8e5c Author: Nick Carboni <ncarboni> AuthorDate: Fri Nov 11 11:40:51 2016 -0500 Commit: Nick Carboni <ncarboni> CommitDate: Fri Nov 11 15:53:46 2016 -0500 Remove the last references to the excludes settings path https://bugzilla.redhat.com/show_bug.cgi?id=1389821 lib/miq_pglogical.rb | 1 - spec/replication/util/miq_pglogical_spec.rb | 6 +----- 2 files changed, 1 insertion(+), 6 deletions(-)
New commit detected on ManageIQ/manageiq/master: https://github.com/ManageIQ/manageiq/commit/d6103a865b7f30c3b21724090bec17dbfe9894d1 commit d6103a865b7f30c3b21724090bec17dbfe9894d1 Author: Nick Carboni <ncarboni> AuthorDate: Fri Nov 11 11:49:08 2016 -0500 Commit: Nick Carboni <ncarboni> CommitDate: Fri Nov 11 15:53:50 2016 -0500 Remove exclude table logic from config activator This is no longer needed because the exclude tables are not stored in the global Settings anymore. https://bugzilla.redhat.com/show_bug.cgi?id=1389821 lib/vmdb/config/activator.rb | 5 ----- 1 file changed, 5 deletions(-)
New commit detected on ManageIQ/manageiq/master: https://github.com/ManageIQ/manageiq/commit/9301b2513065ea103162b1b9f2612d0ecdb6a1fe commit 9301b2513065ea103162b1b9f2612d0ecdb6a1fe Author: Nick Carboni <ncarboni> AuthorDate: Fri Nov 11 15:07:01 2016 -0500 Commit: Nick Carboni <ncarboni> CommitDate: Fri Nov 11 15:53:55 2016 -0500 Set the currently configured excludes to the active ones for new instances This means that `MiqPglogical.new.refresh_excludes` will not change the excluded tables back to the default, but will leave them as they currently are. https://bugzilla.redhat.com/show_bug.cgi?id=1389821 lib/miq_pglogical.rb | 3 ++- spec/replication/util/miq_pglogical_spec.rb | 22 ++++++++++++++++++++++ 2 files changed, 24 insertions(+), 1 deletion(-)
New commit detected on ManageIQ/manageiq/master: https://github.com/ManageIQ/manageiq/commit/0e7910914e9d116507e84f7e97acb6415ae96c13 commit 0e7910914e9d116507e84f7e97acb6415ae96c13 Author: Nick Carboni <ncarboni> AuthorDate: Fri Nov 11 15:51:21 2016 -0500 Commit: Nick Carboni <ncarboni> CommitDate: Mon Nov 14 14:47:15 2016 -0500 Add a migration to remove settings_changes rows for the excludes Because we have removed the excludes from the settings.yml file we should remove the changes made in the database. This will leave current installations of pglogical in the same state they were in (we will read the excludes from the pglogical tables), but will reset new installations created when moving from rubyrep to pglogical. https://bugzilla.redhat.com/show_bug.cgi?id=1389821 ...00_remove_replication_excludes_from_settings.rb | 13 +++++++++++ ...move_replication_excludes_from_settings_spec.rb | 27 ++++++++++++++++++++++ 2 files changed, 40 insertions(+) create mode 100644 db/migrate/20160923000000_remove_replication_excludes_from_settings.rb create mode 100644 spec/migrations/20160923000000_remove_replication_excludes_from_settings_spec.rb
https://github.com/ManageIQ/manageiq/pull/12625
New commit detected on ManageIQ/manageiq/euwe: https://github.com/ManageIQ/manageiq/commit/9c9a45bd3451d70add33ae6d00a513d71d7155e8 commit 9c9a45bd3451d70add33ae6d00a513d71d7155e8 Author: Nick Carboni <ncarboni> AuthorDate: Thu Nov 10 18:07:00 2016 -0500 Commit: Nick Carboni <ncarboni> CommitDate: Mon Nov 14 15:46:52 2016 -0500 Get the default excludes from the new file rather than Settings. This also exposes a new method, `#active_excludes`, to retrieve the current excludes being used by pglogical. This will pull the list of tables directly from the pglogical tables in the database rather than using an intermediate list of settings related to the server. https://bugzilla.redhat.com/show_bug.cgi?id=1389821 lib/miq_pglogical.rb | 24 ++++++++++++++++++------ spec/replication/util/miq_pglogical_spec.rb | 9 +++------ 2 files changed, 21 insertions(+), 12 deletions(-)
New commit detected on ManageIQ/manageiq/euwe: https://github.com/ManageIQ/manageiq/commit/a653540a124066c15f1ebe8f29e565076c3648e1 commit a653540a124066c15f1ebe8f29e565076c3648e1 Author: Nick Carboni <ncarboni> AuthorDate: Thu Nov 10 18:05:31 2016 -0500 Commit: Nick Carboni <ncarboni> CommitDate: Mon Nov 14 15:46:42 2016 -0500 Move the default replication excludes out of the settings.yml file This will allow us to move away from storing this database centric setting as related to the server. https://bugzilla.redhat.com/show_bug.cgi?id=1389821 config/default_replication_exclude_tables.yml | 92 ++++++++++++++++++++++++++ config/settings.yml | 93 --------------------------- 2 files changed, 92 insertions(+), 93 deletions(-) create mode 100644 config/default_replication_exclude_tables.yml
New commit detected on ManageIQ/manageiq/euwe: https://github.com/ManageIQ/manageiq/commit/673096ece535af5f1f8f38c4d1d57a7218e96c18 commit 673096ece535af5f1f8f38c4d1d57a7218e96c18 Author: Nick Carboni <ncarboni> AuthorDate: Fri Nov 11 11:25:55 2016 -0500 Commit: Nick Carboni <ncarboni> CommitDate: Mon Nov 14 15:47:01 2016 -0500 Add class methods for refreshing the excludes from the queue We can't use the instance methods with the queue because this is not an ActiveRecord object. https://bugzilla.redhat.com/show_bug.cgi?id=1389821 lib/miq_pglogical.rb | 14 ++++++++++++++ spec/replication/util/miq_pglogical_spec.rb | 13 +++++++++++++ 2 files changed, 27 insertions(+)
New commit detected on ManageIQ/manageiq/euwe: https://github.com/ManageIQ/manageiq/commit/e499c7607222d44491534a0118e10ffad283dc75 commit e499c7607222d44491534a0118e10ffad283dc75 Author: Nick Carboni <ncarboni> AuthorDate: Fri Nov 11 11:40:51 2016 -0500 Commit: Nick Carboni <ncarboni> CommitDate: Mon Nov 14 15:47:11 2016 -0500 Remove the last references to the excludes settings path https://bugzilla.redhat.com/show_bug.cgi?id=1389821 lib/miq_pglogical.rb | 1 - spec/replication/util/miq_pglogical_spec.rb | 6 +----- 2 files changed, 1 insertion(+), 6 deletions(-)
New commit detected on ManageIQ/manageiq/euwe: https://github.com/ManageIQ/manageiq/commit/fee7f5e00ca29e7cafc71ee68cfc206bb3ee2915 commit fee7f5e00ca29e7cafc71ee68cfc206bb3ee2915 Author: Nick Carboni <ncarboni> AuthorDate: Fri Nov 11 11:49:08 2016 -0500 Commit: Nick Carboni <ncarboni> CommitDate: Mon Nov 14 15:47:20 2016 -0500 Remove exclude table logic from config activator This is no longer needed because the exclude tables are not stored in the global Settings anymore. https://bugzilla.redhat.com/show_bug.cgi?id=1389821 lib/vmdb/config/activator.rb | 5 ----- 1 file changed, 5 deletions(-)
New commit detected on ManageIQ/manageiq/euwe: https://github.com/ManageIQ/manageiq/commit/23ba1a112be5bf19c203926d34424bc672183f4a commit 23ba1a112be5bf19c203926d34424bc672183f4a Author: Nick Carboni <ncarboni> AuthorDate: Fri Nov 11 15:07:01 2016 -0500 Commit: Nick Carboni <ncarboni> CommitDate: Mon Nov 14 15:47:29 2016 -0500 Set the currently configured excludes to the active ones for new instances This means that `MiqPglogical.new.refresh_excludes` will not change the excluded tables back to the default, but will leave them as they currently are. https://bugzilla.redhat.com/show_bug.cgi?id=1389821 lib/miq_pglogical.rb | 3 ++- spec/replication/util/miq_pglogical_spec.rb | 22 ++++++++++++++++++++++ 2 files changed, 24 insertions(+), 1 deletion(-)
UI added in https://github.com/ManageIQ/manageiq/pull/12604
New commit detected on ManageIQ/manageiq/euwe: https://github.com/ManageIQ/manageiq/commit/4fd04ca2f56d19f31921d5f75224da18b2e40c49 commit 4fd04ca2f56d19f31921d5f75224da18b2e40c49 Author: Harpreet Kataria <hkataria> AuthorDate: Wed Nov 16 10:27:43 2016 -0500 Commit: Oleg Barenboim <chessbyte> CommitDate: Wed Nov 16 12:42:02 2016 -0500 Merge pull request #12604 from lgalis/separate_replication_exclude_tables_ui Replication excluded tables UI moved to the replication tab (cherry picked from commit e28727a3cae37d561d8dd7daca87e94726cbddf0) https://bugzilla.redhat.com/show_bug.cgi?id=1389821 app/assets/javascripts/application.js | 1 + .../ops/pglogical_replication_form_controller.js | 57 ++++++++++++++++------ app/assets/javascripts/miq_angular_application.js | 1 + app/controllers/ops_controller/settings/common.rb | 20 ++++++-- app/views/ops/_settings_replication_tab.html.haml | 8 +++ bower.json | 1 + .../ops_controller/settings/common_spec.rb | 18 +++++++ .../pglogical_replication_form_controller_spec.js | 15 ++++-- 8 files changed, 100 insertions(+), 21 deletions(-)