Red Hat Bugzilla – Bug 1167964
RHEL7.1 ipa replica unable to replicate to rhel6 master
Last modified: 2015-03-05 05:15:50 EST
Description of problem: I'm seeing issues with replication between new RHEL7.1 IPA replica added to environment with IPA master on RHEL6.6. [25/Nov/2014:20:49:39 +051800] NSMMReplicationPlugin - multimaster_be_state_change: replica dc=testrelm ,dc=test is coming online; enabling replication [25/Nov/2014:20:49:39 +051800] - Skipping CoS Definition cn=Password Policy,cn=accounts,dc=testrelm,dc= test--no CoS Templates found, which should be added before the CoS Definition. [25/Nov/2014:20:49:41 +051800] NSMMReplicationPlugin - [S] Schema agmt="cn=meTovm-idm-052.testrelm.test " (vm-idm-052:389) must not be overwritten (set replication log for additional info) [25/Nov/2014:20:49:42 +051800] NSMMReplicationPlugin - [S] Schema agmt="cn=meTovm-idm-052.testrelm.test " (vm-idm-052:389) must not be overwritten (set replication log for additional info) [25/Nov/2014:20:49:42 +051800] NSMMReplicationPlugin - agmt="cn=meTovm-idm-052.testrelm.test" (vm-idm-0 52:389): Warning: unable to replicate schema: rc=1 Version-Release number of selected component (if applicable): Master (rhel6.6): ipa-server-3.0.0-42.el6.x86_64 389-ds-base-1.2.11.15-48.el6_6.x86_64 Replica (rhel7.1): ipa-server-4.1.0-7.el7.x86_64 389-ds-base-1.3.3.1-9.el7.x86_64 How reproducible: always Steps to Reproduce: 1. setup IPA master on RHEL6.6 server 2. setup IPA replica on RHEL7.1 server 3. Actual results: errors in log file and Expected results: Additional info: getting more logs and replication debugging information that I will attach.
Created attachment 961321 [details] RHEL 6.6 Master dirsrv errors log after replica install
Created attachment 961322 [details] RHEL 7.1 Replica dirsrv errors log after replica install
Created attachment 961323 [details] RHEL 6.6 Master dirsrv errors log after clean restart with replication logging
Created attachment 961324 [details] RHEL 7.1 Replica dirsrv errors log after clean restart with replication logging
Created attachment 961360 [details] RHEL 6.6 Master dirsrv errors log after copy schema
Created attachment 961361 [details] RHEL 7.1 Replica dirsrv errors log after copy schema
Ludwig or Thierry, can either of you please investigate this one?
I'll look into it
Just few comments: - The problem is related to schema replication between version - During the first replication session the master fail to send its schema master [25/Nov/2014:20:49:34 +051800] NSMMReplicationPlugin - agmt="cn=meTovm-idm-046.testrelm.test" (vm-idm-046:389): Schema replication update failed: Server is unwilling to perform [25/Nov/2014:20:49:34 +051800] NSMMReplicationPlugin - agmt="cn=meTovm-idm-046.testrelm.test" (vm-idm-046:389): Warning: unable to replicate schema: rc=1 consumer [25/Nov/2014:20:49:33 +051800] schema - [C] Local objectClasses must not be overwritten (set replication log for additional info) - The ability for a DS instance to "learn" its missing definition is done in https://fedorahosted.org/389/ticket/47721 - The above errors should be transient during the "learning" phase. So here it is looking like the DS instance are not able to learn the missing definitions. I think enabling replication log should help to know why extra definitions are missing
The error log after schema copy still says that attributes cannot be found in the schema. Can you tar the schema directory on both machines and upload
Looks like replication cannot reconcile the schema differences: [25/Nov/2014:23:37:24 +051800] schema - remote consumer schema attribute [nsViewFilter] syntax can not be overwritten [25/Nov/2014:23:37:25 +051800] schema - remote consumer schema attribute [mgrpRFC822MailMember] syntax can not be overwritten [25/Nov/2014:23:37:26 +051800] NSMMReplicationPlugin - [S] Schema agmt="cn=meTovm-idm-052.testrelm.test" (vm-idm-052:389) must not be overwritten (set replication log for additional info) Thierry, can you confirm that this is a situation where schemaS are incompatible. Scott, please uploade the schema files from both instances
A server acting as a consumer may refuse a received schema if the schema is missing some definitions (attribute/objectclass) compare to the schema currently running on the consumer. In that case the message is logged with a [C]. A server acting as a supplier may refuse to send the schema if the schema it is about to send (its schema) is missing definitions compare to what is currently running on the consumer. In that case the message is logged with a [S] You may enable the replication log on both servers, it will then log additional information in particular what attribute/objectclass are problematic. This kind of issue should be resolved by https://fedorahosted.org/389/ticket/47721 that allows a server to 'learn' what it does not know. So such message should be transient. 47721 needs to be fixed in one or both of the replica to make it work. Apparently there is a corner case here and likely there is a bug in 47721.
Thierry, the snippet in comment 12 is from a log with replication logging, it is in the attachment (4) But copying the schema files to 6.6 should resolve the issue, no ?
In this attachement we can see that 7.1 server did not want to update 6.6 because: [25/Nov/2014:23:36:51 +051800] schema - Attribute nsds5ReplicaPreciseTombstonePurging is not allowed in 'nsDS5Replica' of the remote consumer schema [25/Nov/2014:23:36:53 +051800] schema - Attribute nstombstonecsn is not allowed in 'nsTombstone' of the remote consumer schema [25/Nov/2014:23:36:53 +051800] schema - Attribute nsds5ReplicaProtocolTimeout is not allowed in 'nsDS5ReplicationAgreement' of the remote consumer schema [25/Nov/2014:23:36:53 +051800] schema - Attribute nsSaslMapPriority is not allowed in 'nsSaslMapping' of the remote consumer schema [25/Nov/2014:23:36:53 +051800] schema - Attribute sslVersionMin is not allowed in 'nsEncryptionConfig' of the remote consumer schema [25/Nov/2014:23:36:53 +051800] schema - Attribute nsRoleScopeDN is not allowed in 'nsRoleDefinition' of the remote consumer schema [25/Nov/2014:23:36:54 +051800] schema - Attribute winSyncDirectoryFilter is not allowed in 'nsDSWindowsReplicationAgreement' of the remote consumer schema [25/Nov/2014:23:36:54 +051800] schema - Attribute dnaRemoteBindDN is not allowed in 'dnaPluginConfig' of the remote consumer schema [25/Nov/2014:23:36:57 +051800] schema - Attribute dnaRemoteBindMethod is not allowed in 'dnaSharedConfig' of the remote consumer schema [25/Nov/2014:23:36:58 +051800] schema - Attribute mail is no longer 'required' in 'mailGroup' of the remote consumer schema but is now 'allowed' [25/Nov/2014:23:37:06 +051800] schema - Attribute mail is not allowed in 'mailGroup' of the remote consumer schema [25/Nov/2014:23:37:06 +051800] schema - Attribute ipaAssignedIDView is not allowed in 'ipaHost' of the remote consumer schema [25/Nov/2014:23:37:07 +051800] schema - Attribute ipaNTSIDBlacklistIncoming is not allowed in 'ipaNTTrustedDomain' of the remote consumer schema [25/Nov/2014:23:37:07 +051800] schema - Attribute ipaRangeType is not required in 'ipaIDrange' of the remote consumer schema [25/Nov/2014:23:37:07 +051800] schema - Attribute cn is not allowed in 'idnsRecord' of the remote consumer schema [25/Nov/2014:23:37:08 +051800] schema - Remote idnsRecord schema objectclasses is a superset of the received one. [25/Nov/2014:23:37:08 +051800] schema - Attribute idnsSecInlineSigning is not allowed in 'idnsZone' of the remote consumer schema [25/Nov/2014:23:37:08 +051800] schema - Do not check if this OBJECTCLASS is missing on local/remote schema [printer-uri or printer-uri-oid] [25/Nov/2014:23:37:08 +051800] schema - Attribute ipaSudoRunAsExtUserGroup is not allowed in 'ipaSudoRule' of the remote consumer schema [25/Nov/2014:23:37:09 +051800] schema - Attribute ipaUniqueID is not required in 'ipaSudoCmdGrp' of the remote consumer schema [25/Nov/2014:23:37:09 +051800] schema - Attribute nsds5ReplicaPreciseTombstonePurging is not allowed in 'nsDS5Replica' of the remote consumer schema [25/Nov/2014:23:37:10 +051800] schema - Attribute nstombstonecsn is not allowed in 'nsTombstone' of the remote consumer schema [25/Nov/2014:23:37:10 +051800] schema - Attribute nsds5ReplicaProtocolTimeout is not allowed in 'nsDS5ReplicationAgreement' of the remote consumer schema [25/Nov/2014:23:37:10 +051800] schema - Attribute nsSaslMapPriority is not allowed in 'nsSaslMapping' of the remote consumer schema [25/Nov/2014:23:37:10 +051800] schema - Attribute sslVersionMin is not allowed in 'nsEncryptionConfig' of the remote consumer schema [25/Nov/2014:23:37:11 +051800] schema - Attribute nsRoleScopeDN is not allowed in 'nsRoleDefinition' of the remote consumer schema [25/Nov/2014:23:37:11 +051800] schema - Attribute winSyncDirectoryFilter is not allowed in 'nsDSWindowsReplicationAgreement' of the remote consumer schema [25/Nov/2014:23:37:11 +051800] schema - Attribute dnaRemoteBindDN is not allowed in 'dnaPluginConfig' of the remote consumer schema [25/Nov/2014:23:37:11 +051800] schema - Attribute dnaRemoteBindMethod is not allowed in 'dnaSharedConfig' of the remote consumer schema [25/Nov/2014:23:37:12 +051800] schema - Attribute mail is not allowed in 'mailGroup' of the remote consumer schema [25/Nov/2014:23:37:12 +051800] schema - Attribute ipaAssignedIDView is not allowed in 'ipaHost' of the remote consumer schema [25/Nov/2014:23:37:12 +051800] schema - Attribute ipaNTSIDBlacklistIncoming is not allowed in 'ipaNTTrustedDomain' of the remote consumer schema [25/Nov/2014:23:37:12 +051800] schema - Attribute ipaRangeType is not required in 'ipaIDrange' of the remote consumer schema [25/Nov/2014:23:37:14 +051800] schema - Attribute DLVRecord is not allowed in 'idnsRecord' of the remote consumer schema [25/Nov/2014:23:37:15 +051800] schema - Attribute idnsSecInlineSigning is not allowed in 'idnsZone' of the remote consumer schema [25/Nov/2014:23:37:15 +051800] schema - Attribute ipaSudoRunAsExtUserGroup is not allowed in 'ipaSudoRule' of the remote consumer schema [25/Nov/2014:23:37:21 +051800] schema - Attribute ipaUniqueID is not required in 'ipaSudoCmdGrp' of the remote consumer schema [25/Nov/2014:23:37:24 +051800] schema - remote consumer schema attribute [nsViewFilter] syntax can not be overwritten [25/Nov/2014:23:37:25 +051800] schema - remote consumer schema attribute [mgrpRFC822MailMember] syntax can not be overwritten [25/Nov/2014:23:37:26 +051800] NSMMReplicationPlugin - [S] Schema agmt="cn=meTovm-idm-052.testrelm.test" (vm-idm-052:389) must not be overwritten (set replication log for additional info) Those errors occur regularly. So 47721 fails to resolve the missing definitions. Perhaps in some specific case it needs to be fixed on both side. In those cases, copying the schema should be enough to fix the issue. about user99.ldif, was it also overwritten ?
Created attachment 963381 [details] /etc/dirsrv from new RHEL6.6 master
Created attachment 963382 [details] /etc/dirsrv from RHEL7.1 replica
I've uploaded /etc/dirsrv to look at schema files (and anything else) from two new servers where I reproduced this issue. I also have the logs to these if needed and can provide login access if necessary. Let me know if you need anything else from these?
Upstream ticket: https://fedorahosted.org/freeipa/ticket/4794
Fixed upstream master: https://fedorahosted.org/freeipa/changeset/489dfe64689f86f7ddc4ad0784de0636f8e6c1f8 ipa-4-1: https://fedorahosted.org/freeipa/changeset/2fa07b1d24f61f9bcff5adb804a18c9eae72932d
Verified. Version :: ipa-server-4.1.0-12.el7.x86_64 Results :: [root@rhel7-3 ~]# grep "Attribute cn is not allowed in 'idnsRecord' of the remote consumer schema" /var/log/dirsrv/slapd-EXAMPLE-COM/errors [root@rhel7-3 ~]# So, some of the main errors are not present. [root@rhel6-1 ~]# ipa dnszone-find Zone name: 122.168.192.in-addr.arpa. Authoritative nameserver: rhel6-1.example.com. Administrator e-mail address: hostmaster.example.com. SOA serial: 1418352809 SOA refresh: 3600 SOA retry: 900 SOA expire: 1209600 SOA minimum: 3600 Active zone: TRUE Allow query: any; Allow transfer: none; Zone name: example.com Authoritative nameserver: rhel6-1.example.com. Administrator e-mail address: hostmaster.example.com. SOA serial: 1418357082 SOA refresh: 3600 SOA retry: 900 SOA expire: 1209600 SOA minimum: 3600 Active zone: TRUE Allow query: any; Allow transfer: none; ---------------------------- Number of entries returned 2 ---------------------------- It has to be noted that the first sign I saw there was a problem originally was that the RHEL6 server returned 0 entries for dnszone-find.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2015-0442.html