Red Hat Bugzilla – Bug 1175277
Data replication not working as expected after data restore from full backup
Last modified: 2015-03-05 05:19:00 EST
Created attachment 970085 [details] cotains replica's dirsrv's error log and console output of master Description of problem: I tried following scenario Data restore from full backup on Master ======================================= (1)Added one user(masteruser1) on Master and one user (replicauser1)on Replica before backup and expecting these two users available after data restore from full backup. (2)Took full ipa backup (3)Deleted users added in step(1). Expecting these deleted users to be available after restore (4)Added one more user(masteruser2) on Master and one more user (replicauser2)on Replica after backup and expecting these two users not available after data restore from full backup. (5)Restored data from full backup on Master. After restoration i was expecting users added in step(1) available But i found user added on master before backup (masteruser1) and user added on replica after backup (replicauser2). Version-Release number of selected component (if applicable): [root@dhcp207-214 ~]# rpm -q ipa-server ipa-server-4.1.0-12.el7.x86_64 [root@dhcp207-214 ~]# How reproducible: Always Steps to reproduce. - Mentioned in description. Actual results: Users added on replica before backup not available after data restore from full backup Expected results: Users added on master and replica before backup should be available after data restore from full backup Additional info: (1)Please refer the attachment for replica's dirsrv's errors log and console output of master
Also found this behaviour in following scenario also - Full restore from full backup
Upstream ticket: https://fedorahosted.org/freeipa/ticket/4822
Hi Kaleem, Would you provide the access/errors logs and config (dse.ldif) of both master and replica ? Also in the attachment you said 'users after reinitializing the replication on Replica(first attempt) :' What does it mean ? Thanks thierry
Hello Kaleem, The logs do not contain 'replicauser[12]' or 'masteruser[12]'. I can only find a 'testuser1' in the logs. Would it be possible that you reproduce the problem and then provide the logs and config? If possible enable replication logging on replica/master before the retry MOD (cn=config, nsslapd-errorlog-level: 8192). Thanks for you help Kaleem
Provided machines for debugging on which issue found. Users added through automated scripts are different.
The problem is that once the master is recovered, replication master<-->replica gets broken. BEFORE the replica can get the chance to send all old updates (DEL testuser1, DEL testuser2, ADD testuser3, ADD testuser4), the DNA plugin triggers internal modifications. One of the modification (I ignore if there are others modifications) is the ADD of the entry: dnaHostname=master.testrelm.test+dnaPortNum=389,cn=posix-ids,cn=dna,cn=ipa ,cn=etc,dc=testrelm,dc=test. This modification hit the bug https://fedorahosted.org/389/ticket/47986. This intenal MOD on master has a CSN that is unknown from the replica. So the replica can not update the supplier. A fix is under implementation to remove the RUV from the backup ldif. This because a recovered master should not be updated with updates more recent than the backup.
Fixed upstream master: https://fedorahosted.org/freeipa/changeset/05e6adecb51b93e9b9d2326df4eabee90c3dfe72 ipa-4-1: https://fedorahosted.org/freeipa/changeset/eb7917026d418a6d6a1e7a24a19097065df10497
*** Bug 1182568 has been marked as a duplicate of this bug. ***
A regression was found in the fix.
Regression fixed upstream master: https://fedorahosted.org/freeipa/changeset/4994cd1d8dde69d94130894ca86aa8a11fb590bf ipa-4-1: https://fedorahosted.org/freeipa/changeset/85599416ae4a0cc83b8e0669e686740dd2a10ea2
Verified. IPA Version: ============ [root@master ~]# rpm -q ipa-server ipa-server-4.1.0-16.el7.x86_64 [root@master ~]# [root@master ~]# ipa-restore -p xxxxxxxx --data -U /var/lib/ipa/backup/ipa-data-2015-01-28-15-42-55/ Preparing restore from /var/lib/ipa/backup/ipa-data-2015-01-28-15-42-55/ on master.testrelm.test Performing DATA restore from DATA backup Each master will individually need to be re-initialized or re-created from this one. The replication agreements on masters running IPA 3.1 or earlier will need to be manually re-enabled. See the man page for details. Disabling all replication. Disabling replication agreement on dhcp207-58.testrelm.test to master.testrelm.test Stopping Directory Server Restoring from userRoot in TESTRELM-TEST Restoring from ipaca in TESTRELM-TEST Starting Directory Server The ipa-restore command was successful [root@master ~]# ipa user-find --------------- 3 users matched --------------- User login: admin Last name: Administrator Home directory: /home/admin Login shell: /bin/bash UID: 381600000 GID: 381600000 Account disabled: False Password: True Kerberos keys available: True User login: testuser1 First name: testuser1 Last name: testuser1 Home directory: /home/testuser1 Login shell: /bin/sh Email address: testuser1@testrelm.test UID: 381600001 GID: 381600001 Account disabled: False Password: True Kerberos keys available: True User login: testuser2 First name: testuser2 Last name: testuser2 Home directory: /home/testuser2 Login shell: /bin/sh Email address: testuser2@testrelm.test UID: 381700500 GID: 381700500 Account disabled: False Password: True Kerberos keys available: True ---------------------------- Number of entries returned 3 ---------------------------- [root@master ~]# sleep 300 [root@master ~]# ipa user-find --------------- 3 users matched --------------- User login: admin Last name: Administrator Home directory: /home/admin Login shell: /bin/bash UID: 381600000 GID: 381600000 Account disabled: False Password: True Kerberos keys available: True User login: testuser1 First name: testuser1 Last name: testuser1 Home directory: /home/testuser1 Login shell: /bin/sh Email address: testuser1@testrelm.test UID: 381600001 GID: 381600001 Account disabled: False Password: True Kerberos keys available: True User login: testuser2 First name: testuser2 Last name: testuser2 Home directory: /home/testuser2 Login shell: /bin/sh Email address: testuser2@testrelm.test UID: 381700500 GID: 381700500 Account disabled: False Password: True Kerberos keys available: True ---------------------------- Number of entries returned 3 ----------------- Please find the attached console output of Master/Replica.
Created attachment 985004 [details] console output
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2015-0442.html