Bug 663752 - Cert renewal for attrcrypt and encchangelog
Summary: Cert renewal for attrcrypt and encchangelog
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: 389
Classification: Retired
Component: Security - SSL
Version: 1.2.7
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: ---
Assignee: Noriko Hosoi
QA Contact: Viktor Ashirov
URL:
Whiteboard:
Depends On:
Blocks: 434915 389_1.2.9 713317 713318
TreeView+ depends on / blocked
 
Reported: 2010-12-16 18:41 UTC by Noriko Hosoi
Modified: 2015-12-07 16:44 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 713317 713318 (view as bug list)
Environment:
Last Closed: 2015-12-07 16:44:56 UTC


Attachments (Terms of Use)
git patch file (master) (38.35 KB, patch)
2011-01-12 02:48 UTC, Noriko Hosoi
nkinder: review+
Details | Diff
git patch file (master) (5.85 KB, patch)
2011-06-14 23:20 UTC, Noriko Hosoi
nhosoi: review?
rmeggins: review+
Details | Diff
git patch file (master) (1.75 KB, patch)
2011-06-16 19:19 UTC, Noriko Hosoi
nhosoi: review?
rmeggins: review+
Details | Diff
git patch file (master) (2.93 KB, patch)
2011-07-29 00:24 UTC, Noriko Hosoi
nhosoi: review?
rmeggins: review+
Details | Diff

Description Noriko Hosoi 2010-12-16 18:41:15 UTC
Description of problem:
Email discussion:
===========================
Basic Cert renewal scenario
===========================
Assume the server already sets up the both attrcrypt and encchangelog.
0. Before the cert is renewed, customers is supposed to do export the DB contents (db2ldif -E ...).

    # ./db2ldif -n backend_instance -E
    Exported ldif file: /var/lib/dirsrv/slapd-server_ID/ldif/server_ID-backend_instance-date.ldif

1. Stop the server

    service dirsrv stop server_ID

2. Renew the server cert.

3. Edit /etc/dirsrv/slapd-ID/dse.ldif
3-1. Remove attrcrypt entries:

    dn: cn=3DES,cn=encrypted attribute keys,cn=backend_instance,cn=ldbm database,cn=plugins,cn=config
    objectClass: top
    objectClass: extensibleObject
    cn: 3DES
    nsSymmetricKey:: EGXb3F59jqZzOs6gU/MmuP1FS6uesD9i1Y3riJjLsYrEDVloTd98lmOcKsg1
     u0uR8qT4uEnfaHnZY1Y7F4tcaaP9sQiiM2V9BfDBT9e19AqRHAO5vt2LrujT5Js4ZoO0lPXl8o3v
     VaH4Ju10wWs6jSoTyKSzGSHFWd/nrZq0wLg=

    dn: cn=AES,cn=encrypted attribute keys,cn=backend_instance,cn=ldbm database,cn=plugins ,cn=config
    objectClass: top
    objectClass: extensibleObject
    cn: AES
    nsSymmetricKey:: CdPY1U7A6rrt1mgPy60D+0MZ4Am8EAKEp/DL9uXJrdvpBO0+oiH6IxNDtieR
     nFjyG2uKEdy2gwcUGLU3TXWOoqwfANIP1HPWmpWc1nQ6aqMh2OJqvnZhL3j+kkDCDV6y68YyYrK2
     16VJiWEsWnENXqnqpp8bnUQhmo9O1PlU8eI=

3-2. Remove nsSymmetricKey from cn=changelog5,cn=config

    dn: cn=changelog5,cn=config
    [...]
    nsSymmetricKey:: JP4WSyHmkdae7pzfhoCppdk2+09o+i2KTV3CYqjb00z9igQwNf+24v5jh2I7
     UFod/YJtMpuWN/awTkrXGsCWPR3lZEHTWlNG+peXcFVAV2hWTrcPPjXMlsL++wS7urzpW/BFD10v
     kK7WEtzD50iHS0BMxYvWVzyblDk6jybOLD4=

4. Start the server to generate a new symmetric key.

    service dirsrv start server_ID

5. Run online import

    ./ldif2db.pl -D 'cn=directory manager' -w password -n backend_instance -E -i /var/lib/dirsrv/slapd-server_ID/ldif/server_ID-backend_instance-date.ldif

or stop the server then run standalone import.

    service dirsrv stop server_ID
    ./ldif2db -n backend_instance -E -i /var/lib/dirsrv/slapd-server_ID/ldif/server_ID-backend_instance-date.ldif

6. Regenerate the changelog.
I did once removing a replica and recreated it.  Could there be any easier/better way for regenerating a changelog?

=====================
Rich Megginson wrote:
=====================
Sounds like we need a way to either dump/import a changelog or "re-encrypt" it in place.

Comment 1 Noriko Hosoi 2010-12-23 19:49:46 UTC
We already have a replication task to export the changelog:

$ ldapmodify ...
dn: cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config
changetype: modify
add: nsds5Task
nsds5Task: CL2LDIF

It dumps the contents of changelog into the ldif file.
# pwd
/var/lib/dirsrv/slapd-ID/changelogdb
# ls
b50f2982-1dd111b2-9c21b4e8-fd670000_4d13a124000000010000.db4  b50f2982-1dd111b2-9c21b4e8-fd670000.sema
b50f2982-1dd111b2-9c21b4e8-fd670000.ldif
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Snippet of the ldif:
changetype: add
replgen: 4d13a124000000010000
csn: 4d13a1b8000000020000
nsuniqueid: 1cc57481-1dd211b2-91a3b842-4b530000
parentuniqueid: b2133983-1dd111b2-825ea6cb-4a0f0000
dn: uid=s1user0,ou=People,dc=example,dc=com
change:: YWRkOiB1aWQKdWlkOiBzMXVzZXIwCi0KYWRkOiBnaXZlbk5hbWUKZ2l2ZW5OYW1lOiBz
 MQotCmFkZDogb2JqZWN0Q2xhc3MKb2JqZWN0Q2xhc3M6IHRvcApvYmplY3RDbGFzczogcGVyc29u
 Cm9iamVjdENsYXNzOiBvcmdhbml6YXRpb25hbFBlcnNvbgpvYmplY3RDbGFzczogaW5ldG9yZ3Bl
 cnNvbgotCmFkZDogc24Kc246IHVzZXIwCi0KYWRkOiBjbgpjbjogczEgdXNlcjAKLQphZGQ6IHVz
 ZXJQYXNzd29yZAp1c2VyUGFzc3dvcmQ6IHtTU0hBfS9oQnY0N2dYSk1wZlZEb2E1bXZKaUxpQk1p
 cFZWTjdHVEtldmR3PT0KLQphZGQ6IGNyZWF0b3JzTmFtZQpjcmVhdG9yc05hbWU6IHVpZD1hZG1p
 bixvdT1hZG1pbmlzdHJhdG9ycyxvdT10b3BvbG9neW1hbmFnZW1lbnQsbz1uZXRzY2FwZXJvb3QK
 LQphZGQ6IG1vZGlmaWVyc05hbWUKbW9kaWZpZXJzTmFtZTogdWlkPWFkbWluLG91PWFkbWluaXN0
 cmF0b3JzLG91PXRvcG9sb2d5bWFuYWdlbWVudCxvPW5ldHNjYXBlcm9vCiB0Ci0KYWRkOiBjcmVh
 dGVUaW1lc3RhbXAKY3JlYXRlVGltZXN0YW1wOiAyMDEwMTIyMzE5MjMzNVoKLQphZGQ6IG1vZGlm
 eVRpbWVzdGFtcAptb2RpZnlUaW1lc3RhbXA6IDIwMTAxMjIzMTkyMzM1WgotCmFkZDogbnNVbmlx
 dWVJZApuc1VuaXF1ZUlkOiAxY2M1NzQ4MS0xZGQyMTFiMi05MWEzYjg0Mi00YjUzMDAwMAotCmFk
 ZDogdW5oYXNoZWQjdXNlciNwYXNzd29yZAp1bmhhc2hlZCN1c2VyI3Bhc3N3b3JkOiBzMXVzZXIw
 Ci0K

Base64 decoded result:
add: uid
uid: s1user0
-
add: givenName
givenName: s1
-
add: objectClass
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetorgperson
-
add: sn
sn: user0
-
add: cn
cn: s1 user0
-
add: userPassword
userPassword: {SSHA}/hBv47gXJMpfVDoa5mvJiLiBMipVVN7GTKevdw==
-
add: creatorsName
creatorsName: uid=admin,ou=administrators,ou=topologymanagement,o=netscaperoot
-
add: modifiersName
modifiersName: uid=admin,ou=administrators,ou=topologymanagement,o=netscaperoo
 t
-
add: createTimestamp
createTimestamp: 20101223192335Z
-
add: modifyTimestamp
modifyTimestamp: 20101223192335Z
-
add: nsUniqueId
nsUniqueId: 1cc57481-1dd211b2-91a3b842-4b530000
-
add: unhashed#user#password
unhashed#user#password: s1user0
-

Note: The original changelog is encrypted with AES.  The exported ldif is not.

Comment 3 Noriko Hosoi 2011-01-12 02:48:34 UTC
Created attachment 472939 [details]
git patch file (master)

Description: In fixing Bug 182507, the feature to encrypt changelogs
had been introduced. The changelog encryption depends on the server
certificate as the attrcrypt does.  When the server certificate is
renewed, the encrypted changelog won't be decrypted.  This patch
implements/completes the feature to export and import the contents
of the changelog.

See also this section for the steps to export/import changelogs.
http://directory.fedoraproject.org/wiki/Changelog_Encryption#Steps_for_Certificate_Renewal

Comment 4 Noriko Hosoi 2011-01-12 17:40:02 UTC
Reviewed by Nathan (Thank you!!!)

Pushed to master.

$ git merge 663752
Updating 66a666c..ad3c528
Fast-forward
 ldap/servers/plugins/replication/cl5_api.c         |  277 +++++++++++++---
 ldap/servers/plugins/replication/cl5_clcache.c     |   15 +-
 .../plugins/replication/repl5_replica_config.c     |  345 +++++++++++++++++++-
 ldap/servers/plugins/replication/repl5_ruv.c       |    7 +-
 ldap/servers/slapd/slapi-private.h                 |    1 +
 5 files changed, 576 insertions(+), 69 deletions(-)

$ git push
Counting objects: 23, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (12/12), done.
Writing objects: 100% (12/12), 5.77 KiB, done.
Total 12 (delta 10), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
   66a666c..ad3c528  master -> master

Comment 7 Amita Sharma 2011-06-08 09:54:23 UTC
Following http://directory.fedoraproject.org/wiki/Changelog_Encryption#Steps_for_Certificate_Renewal
=============================
This is part one tested OK
============================
1. Preparation
1-1. Install 3 servers with at least 2 backend databases to replicate.
   e.g., suffix "dc=example,dc=com", "dc=test,dc=com"
1-2. Setup SSL on Master servers.
1-3. Setup Master 1 <--> Master 2
               |
               v
           Read only replica

20100 <--->20102

1-4. Stop Master servers and set nsslapd-encryptionalgorithm.  The allowed value is AES or 3DES.
   dn: cn=changelog5,cn=config
   [...]
   nsslapd-encryptionalgorithm: AES

[root@rhel61-ds90-amita ~]# /usr/lib64/dirsrv/slapd-M1/stop-slapd 
[root@rhel61-ds90-amita ~]# service dirsrv status
dirsrv M1 is stopped
dirsrv M2 (pid 1342) is running...
objectClass: top
objectClass: extensibleobject
cn: changelog5
nsslapd-changelogdir: /var/lib/dirsrv/slapd-M1/db/changelog
nsslapd-encryptionalgorithm: AES
creatorsName: cn=directory manager
modifiersName: cn=directory manager
createTimestamp: 20110607111531Z
modifyTimestamp: 20110607111531Z

1-5. Restart Master servers, and initialialize replicas on each agreement on Master 1.
/usr/lib64/dirsrv/slapd-M1/start-slapd

1-6. Verify the replication topology is correctly set up by adding at least one entry to each backend on Master servers.
[root@rhel61-ds90-amita ~]# ldapadd -x -h localhost -p 20100 -D "cn=Directory Manager" -w Secret123  << EOF
dn: uid=amsharma2,ou=people,dc=replsuffix,dc=com
cn: ams
sn: ams
givenname: ams
objectclass: top
objectclass: person
objectclass: organizationalPerson
objectclass: inetOrgPerson
uid: ams
mail: ams@example.com
userpassword: amsamsams
EOF

adding new entry "uid=amsharma2,ou=people,dc=replsuffix,dc=com"

[root@rhel61-ds90-amita ~]# ldapsearch -x -p 20102 -h localhost -D "cn=Directory Manager" -w Secret123 -b "dc=replsuffix,dc=com" | grep amsharma2
# amsharma2, People, replsuffix.com
dn: uid=amsharma2,ou=People,dc=replsuffix,dc=com
uid: amsharma2

1-7. Dump Master servers' changelog to confirm the changelogs are encrypted.
     One changelog per replica; 
     If you set up replicas on 2 backends, e.g., dc=example,dc=com and dc=test,dc=com, there are 2 changelog db files.
     # dbscan -f /var/lib/dirsrv/slapd-master[12]/changelogdb/[...].db4
dbscan -f bb036a82-90f711e0-a47e8da0-ec4827e4_4dee08e2000000010000.db
dbid: 4def39fb000000010000
	replgen: 1307523578 Wed Jun  8 14:29:38 2011
	csn: 4def39fb000000010000
	uniqueid: 8651ea81-91ad11e0-a47e8da0-ec4827e4
	parentuniqueid: 5b0cc403-90f711e0-9a609c91-c5a5a6e6
	dn: uid=amsharma2,ou=people,dc=replsuffix,dc=com
	operation: add
		cn: g????Y?|f?}?:;U
		sn: g????Y?|f?}?:;U
		givenName: g????Y?|f?}?:;U
		objectClass: ??v?G[q>&U
		objectClass:  a?n??z?z???}
		objectClass: ??[?F?L?@?Q????C??
?1??                                           ??
		objectClass: 
??2?/9?q????
		uid: g????Y?|f?}?:;U
?Ga?? 	uid: ?b2???
		mail: ?e????~?vi?
c??(?GX??=QF??o?l{erPassword: ??n?s?%\??b
???cYE?Z{?82??0pn?{?8{rsName: ?{?F?=k
???cYE?Z{?82??0pn?{?8{ersName: ?{?F?=k
		createTimestamp: 
X??V?F???2??
		modifyTimestamp: 
X??V?F???2??
??J?|????0     nsUniqueId: ??e-X?]??XH???r)??dn
           ?d.Yq?	???B
		unhashed#user#password: c?*?[8T%sV9?

1-8. Run some modification ops.  E.g.,
   $ infadd -p <Master 1 port> -s "dc=example,dc=com" -u "cn=directory manager" -w <password>
   $ infadd -p <Master 2 port> -s "dc=test,dc=com" -u "cn=directory manager" -w <password>
=====================================================
This is part two but I am facing the server crash here -
========================================================
[root@rhel61-ds90-amita changelog]# ldapmodify -D "cn=directory manager" -w Secret123 -p 20100 -x -h localhost << EOF
> dn: cn=replica,cn=dc\3Dreplsuffix\2Cdc\3Dcom,cn=mapping tree,cn=config
> changetype: modify
> add: nsds5Task
> nsds5Task: CL2LDIF
> EOF
modifying entry "cn=replica,cn=dc\3Dreplsuffix\2Cdc\3Dcom,cn=mapping tree,cn=config"
ldap_result: Can't contact LDAP server (-1)

[root@rhel61-ds90-amita changelog]# ldapmodify -x -h localhost -p 20100 -D "cn=directory manager" -w Secret123 << EOF
> dn: cn=replica,cn=dc\3Dreplsuffix\2Cdc\3Dcom,cn=mapping tree,cn=config
> changetype: modify
> add: nsds5Task
> nsds5Task: CL2LDIF
> EOF
ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1)
[root@rhel61-ds90-amita changelog]# service dirsrv status
dirsrv M1 dead but pid file exists
dirsrv M2 (pid 1342) is running...
dirsrv M3 (pid 1422) is running...
dirsrv M4 (pid 1507) is running...
dirsrv rhel61-ds90-amita (pid 1580) is running...
[root@rhel61-ds90-amita changelog]# /usr/lib64/dirsrv/slapd-M1/stop-slapd 
Server not running
[root@rhel61-ds90-amita changelog]# /usr/lib64/dirsrv/slapd-M1/start-slapd 
[root@rhel61-ds90-amita changelog]# service dirsrv status
dirsrv M1 (pid 1973) is running...
dirsrv M2 (pid 1342) is running...
dirsrv M3 (pid 1422) is running...
dirsrv M4 (pid 1507) is running...
dirsrv rhel61-ds90-amita (pid 1580) is running...
[root@rhel61-ds90-amita changelog]# ldapmodify -x -h localhost -p 20100 -D "cn=directory manager" -w Secret123 << EOF
> dn: cn=replica,cn=dc\3Dreplsuffix\2Cdc\3Dcom,cn=mapping tree,cn=config
> changetype: modify
> add: nsds5Task
> nsds5Task: CL2LDIF
> EOF
modifying entry "cn=replica,cn=dc\3Dreplsuffix\2Cdc\3Dcom,cn=mapping tree,cn=config"
ldap_result: Can't contact LDAP server (-1)

[root@rhel61-ds90-amita changelog]# service dirsrv status
dirsrv M1 dead but pid file exists
dirsrv M2 (pid 1342) is running...
dirsrv M3 (pid 1422) is running...
dirsrv M4 (pid 1507) is running...
dirsrv rhel61-ds90-amita (pid 1580) is running...
[root@rhel61-ds90-amita changelog]# 
=======
LOGS
=========
[root@rhel61-ds90-amita changelog]# tail -f /var/log/dirsrv/slapd-M1/errors
[08/Jun/2011:14:39:57 +051800] NSMMReplicationPlugin - changelog program - _cl5ReadMod: decrypting "createTimestamp: 32" failed
[08/Jun/2011:14:39:57 +051800] attrcrypt - _back_crypt_crypto_op failed on cipher AES : -8188 - security library has experienced an input length error.
[08/Jun/2011:14:39:57 +051800] attrcrypt - _back_crypt_crypto_op failed on cipher AES : -8188 - security library has experienced an input length error.
[08/Jun/2011:14:39:57 +051800] NSMMReplicationPlugin - changelog program - _cl5ReadMod: decrypting "modifyTimestamp: 32" failed
[08/Jun/2011:14:39:57 +051800] attrcrypt - _back_crypt_crypto_op failed on cipher AES : -8188 - security library has experienced an input length error.
[08/Jun/2011:14:39:57 +051800] attrcrypt - _back_crypt_crypto_op failed on cipher AES : -8188 - security library has experienced an input length error.
[08/Jun/2011:14:39:57 +051800] NSMMReplicationPlugin - changelog program - _cl5ReadMod: decrypting "nsUniqueId: 65" failed
[08/Jun/2011:14:39:57 +051800] attrcrypt - _back_crypt_crypto_op failed on cipher AES : -8188 - security library has experienced an input length error.
[08/Jun/2011:14:39:57 +051800] attrcrypt - _back_crypt_crypto_op failed on cipher AES : -8188 - security library has experienced an input length error.
[08/Jun/2011:14:39:57 +051800] NSMMReplicationPlugin - changelog program - _cl5ReadMod: decrypting "unhashed#user#password: 61" failed

[root@rhel61-ds90-amita changelog]# tail -f /var/log/dirsrv/slapd-M1/access
[08/Jun/2011:14:36:03 +051800] conn=7 op=15 EXT oid="2.16.840.1.113730.3.5.12"
[08/Jun/2011:14:36:03 +051800] conn=7 op=15 RESULT err=0 tag=120 nentries=0 etime=0
[08/Jun/2011:14:36:03 +051800] conn=7 op=16 EXT oid="2.16.840.1.113730.3.5.5" name="Netscape Replication End Session"
[08/Jun/2011:14:36:03 +051800] conn=7 op=16 RESULT err=0 tag=120 nentries=0 etime=0
[08/Jun/2011:14:37:00 +051800] conn=9 op=16 UNBIND
[08/Jun/2011:14:37:00 +051800] conn=9 op=16 fd=66 closed - U1
[08/Jun/2011:14:37:02 +051800] conn=8 op=18 UNBIND
[08/Jun/2011:14:37:02 +051800] conn=8 op=18 fd=65 closed - U1
[08/Jun/2011:14:37:04 +051800] conn=7 op=17 UNBIND
[08/Jun/2011:14:37:04 +051800] conn=7 op=17 fd=64 closed - U1

Please guide.
-Amita

Comment 8 Noriko Hosoi 2011-06-10 19:23:11 UTC
Amita,

Thanks for the good test case.  I could use your VM 10.16.98.212 to investigate your crash problem.

Let me confirm one thing.  On M1, have you renewed the certificate?  The CL2LDIF task shows lots of errors like this.  It looks decrypting the changelogs are all failing on M1 and not sure what's happening...
[..] attrcrypt - _back_crypt_crypto_op failed on cipher AES : -8188 - security library has experienced an input length error.

I've installed my local build.  I had to wipe out the existing changelog and add some changes to the server.  Now, I can successfully export the changelog like this.

# ldapmodify -x -h localhost -p 20100 -D "cn=directory manager" -w Secret123
dn: cn=replica,cn=dc\3Dreplsuffix\2Cdc\3Dcom,cn=mapping tree,cn=config
changetype: modify
add: nsds5Task
nsds5Task: CL2LDIF

modifying entry "cn=replica,cn=dc\3Dreplsuffix\2Cdc\3Dcom,cn=mapping tree,cn=config"

Could you repeat what you did to reproduce the crash and see how it goes with my local build?

Since there's a crash issue, I'm changing the status to ASSIGNED.

Comment 14 Noriko Hosoi 2011-06-14 23:20:35 UTC
Created attachment 504780 [details]
git patch file (master)

Description: When changelog is encrypted and the certificate
used for the encryption has a problem (e.g., expired, renewed,
etc.), running the CL2LDIF task could crash the server.  This
patch is adding more error checks for the decrypted result.
If a problem is found, it skips the change.

Note: this problem was found by Amita's verification effort.
Thanks, Amita!

Comment 15 Noriko Hosoi 2011-06-15 00:17:48 UTC
Reviewed by Rich (Thank you!!)

Pushed to master.

$ git merge 663752
Updating 9cd0752..c35e240
Fast-forward
 ldap/servers/plugins/replication/cl5_api.c |   36 +++++++++++++++++++++------
 ldap/servers/slapd/util.c                  |   20 +++++++++++----
 2 files changed, 42 insertions(+), 14 deletions(-)

$ git push
Counting objects: 17, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (9/9), done.
Writing objects: 100% (9/9), 1.31 KiB, done.
Total 9 (delta 7), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
   9cd0752..c35e240  master -> master

pushed to RHEL-6:
(cherry picked from commit c35e240ff8a65225b8e1f890ccdc54da7533dbcf)

$ git cherry-pick c35e240ff8a65225b8e1f890ccdc54da7533dbcf
[RHEL-6 cad7963] Bug 663752 - Cert renewal for attrcrypt and encchangelog
 2 files changed, 42 insertions(+), 14 deletions(-)

$ git push redhat RHEL-6
Counting objects: 17, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (9/9), done.
Writing objects: 100% (9/9), 1.36 KiB, done.
Total 9 (delta 7), reused 0 (delta 0)
To ssh://git.engineering.redhat.com/srv/git/users/rmeggins/ds.git
   4312b7a..da4c064  RHEL-6 -> RHEL-6

Comment 16 Noriko Hosoi 2011-06-15 00:22:18 UTC
pushed to ds-replication-RHEL-6, as well.
(cherry picked from commit c35e240ff8a65225b8e1f890ccdc54da7533dbcf)

$ git cherry-pick c35e240ff8a65225b8e1f890ccdc54da7533dbcf
[ds-replication-RHEL-6 5984448] Bug 663752 - Cert renewal for attrcrypt and encchangelog
 2 files changed, 42 insertions(+), 14 deletions(-)

$ git push redhat ds-replication-RHEL-6
Counting objects: 17, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (9/9), done.
Writing objects: 100% (9/9), 1.36 KiB, done.
Total 9 (delta 7), reused 0 (delta 0)
To ssh://git.engineering.redhat.com/srv/git/users/rmeggins/ds.git
   80f966a..9ecc6ff  ds-replication-RHEL-6 -> ds-replication-RHEL-6

Comment 20 Noriko Hosoi 2011-06-16 19:19:30 UTC
Created attachment 505132 [details]
git patch file (master)

Description: There was a bug in using openldap API ldif_read_record
in cl5ImportLDIF (replication/cl5_api.c).  The API ldif_read_record
reuses the buffer allocated internally, where buffer length variable
plays a role to determine the current buffer is large enough for the
new ldif line.  The caller function cl5ImportLDIF freed the buffer
without setting 0 to the length.  It caused segfault.
This patch sets 0 to the buffer length when the buffer is freed.

Comment 21 Noriko Hosoi 2011-06-16 20:13:52 UTC
Reviewed by Rich (Thank you!)

Pushed to master.

$ git merge 663752
Updating c35e240..26695d3
Fast-forward
 ldap/servers/plugins/replication/cl5_api.c |   13 +++++++++----
 1 files changed, 9 insertions(+), 4 deletions(-)
$ git push
Counting objects: 13, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (7/7), done.
Writing objects: 100% (7/7), 957 bytes, done.
Total 7 (delta 5), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
   c35e240..26695d3  master -> master

ds-replication-RHEL-6:
(cherry picked from commit 26695d3fd9cab800b141f67353e400ae65ed732e)

$ git cherry-pick 26695d3fd9cab800b141f67353e400ae65ed732e
[ds-replication-RHEL-6 fbe5f6c] Bug 663752 - Cert renewal for attrcrypt and encchangelog
 1 files changed, 9 insertions(+), 4 deletions(-)

$ git push redhat ds-replication-RHEL-6
Counting objects: 13, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (7/7), done.
Writing objects: 100% (7/7), 999 bytes, done.
Total 7 (delta 5), reused 0 (delta 0)
To ssh://git.engineering.redhat.com/srv/git/users/rmeggins/ds.git
   9ecc6ff..275ab83  ds-replication-RHEL-6 -> ds-replication-RHEL-6

Comment 25 Noriko Hosoi 2011-07-29 00:24:11 UTC
Created attachment 515807 [details]
git patch file (master)

Description: Replica config modify callback replica_config_post_
modify (repl5_replica_config.c) calls an internal modify API with
s_configLock held.  The modify ends up calling a replica config
callback, in which it tries to acquire the same s_configLock and
it hangs there since the locking function PR_Lock is not re-entrant.
This patch avoids calling the internal modify API inside of s_configLock.

Comment 27 Noriko Hosoi 2011-08-01 16:51:07 UTC
Reviewed by Rich (Thanks!!).

Pushed to master.

$ git am `pwd`/0001-Bug-663752-Cert-renewal-for-attrcrypt-and-encchangel.patch
Applying: Bug 663752 - Cert renewal for attrcrypt and encchangelog
/export/src/ds90/ldapserver/ldapserver/.git/rebase-apply/patch:63: trailing whitespace.
    
$ git push
Counting objects: 13, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (7/7), done.
Writing objects: 100% (7/7), 1.16 KiB, done.
Total 7 (delta 5), reused 0 (delta 0)
To ssh://git.fedorahosted.org/git/389/ds.git
   7a0548b..a5fdcdd  master -> master

Pushed to ds-replication-RHEL-6, as well.

(cherry picked from commit a5fdcddd4acc0811faa4a0152218a9702c979115)

$ git push redhat ds-replication-RHEL-6
nhosoi@git.engineering.redhat.com's password: 
Counting objects: 13, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (7/7), done.
Writing objects: 100% (7/7), 1.20 KiB, done.
Total 7 (delta 5), reused 0 (delta 0)
To ssh://git.engineering.redhat.com/srv/git/users/rmeggins/ds.git
   adb1753..fae50d6  ds-replication-RHEL-6 -> ds-replication-RHEL-6

Note: this fix is in ds-replication-1.2.8.7 or newer.

Comment 28 Amita Sharma 2011-09-22 11:33:11 UTC
1. Preparation
1-1. Install 3 servers with at least 2 backend databases to replicate.
   e.g., suffix "dc=example,dc=com", "dc=test,dc=com"
1-2. Setup SSL on Master servers.
1-3. Setup Master 1 <--> Master 2
               |
               v
           Read only replica
1-4. Stop Master servers and set nsslapd-encryptionalgorithm.  The allowed value is AES or 3DES.
   dn: cn=changelog5,cn=config
   [...]
   nsslapd-encryptionalgorithm: AES
=================================================================
[root@snmaptest /]# /usr/lib64/dirsrv/slapd-M1/stop-slapd 
[root@snmaptest /]# service dirsrv status
dirsrv M1 is stopped
dirsrv M2 (pid 15253) is running...

dn: cn=changelog5,cn=config
objectClass: top
objectClass: extensibleobject
cn: changelog5
nsslapd-changelogdir: /var/lib/dirsrv/slapd-M1/db/changelog
creatorsName: cn=directory manager
modifiersName: cn=directory manager
createTimestamp: 20110922090836Z
modifyTimestamp: 20110922090836Z
nsslapd-encryptionalgorithm: AES
=================================================================

1-5. Restart Master servers, and initialialize replicas on each agreement on Master 1.
=======================================================================================
[root@snmaptest ~]# ldapmodify -x -h localhost -p 30100 -D "cn=Directory Manager" -w Secret123  << EOF
> dn: cn=M1_to_M2,cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config
> changetype: modify
> replace: nsds5beginreplicarefresh
> nsds5beginreplicarefresh: start
> EOF
modifying entry "cn=M1_to_M2,cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config"

[root@snmaptest ~]# ldapmodify -x -h localhost -p 30100 -D "cn=Directory Manager" -w Secret123  << EOF
dn: cn=M1_to_M4,cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config
changetype: modify
replace: nsds5beginreplicarefresh
nsds5beginreplicarefresh: start
EOF

modifying entry "cn=M1_to_M4,cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config"
=======================================================================================

1-6. Verify the replication topology is correctly set up by adding at least one entry to each backend on Master servers.
==============================================================================================================
[root@snmaptest ~]# ldapadd -x -h localhost -p 30100 -D "cn=Directory Manager" -w Secret123  << EOF
> dn: uid=amsharma2,ou=people,dc=example,dc=com
> cn: ams
> sn: ams
> givenname: ams
> objectclass: top
> objectclass: person
> objectclass: organizationalPerson
> objectclass: inetOrgPerson
> uid: ams
> mail: ams@example.com
> userpassword: amsamsams
> EOF
adding new entry "uid=amsharma2,ou=people,dc=example,dc=com"

[root@snmaptest ~]# ldapsearch -x -p 30102 -h localhost -D "cn=Directory Manager" -w Secret123 -b "dc=example,dc=com" | grep amsharma2
# amsharma2, People, example.com
dn: uid=amsharma2,ou=People,dc=example,dc=com
uid: amsharma2
==============================================================================================================

1-7. Dump Master servers' changelog to confirm the changelogs are encrypted.
     One changelog per replica; 
     If you set up replicas on 2 backends, e.g., dc=example,dc=com and dc=test,dc=com, there are 2 changelog db files.
     # dbscan -f /var/lib/dirsrv/slapd-master[12]/changelogdb/[...].db4
==============================================================================================================
[root@snmaptest changelog]# dbscan -f 6e06a173-e4fa11e0-9528d0a9-2fb8bf08_4e7afb14000000010000.db4

dbid: 4e7b0243000000010000
	replgen: 1316684355 Thu Sep 22 15:09:15 2011
	csn: 4e7b0243000000010000
	uniqueid: a4461f82-e4fe11e0-9528d0a9-2fb8bf08
	parentuniqueid: 6cd57425-e4fa11e0-9528d0a9-2fb8bf08
	dn: uid=amsharma2,ou=people,dc=example,dc=com
	operation: add
		cn: ?	?H?2B,?g?e<?
		sn: ?	?H?2B,?g?e<?
		givenName: ?	?H?2B,?g?e<?
		objectClass: 
?u???nB1??4
?Z??Ww_0g\      objectClass: i?]
4?,??;?.e??     objectClass: ??)?b?H)-??\4P
		objectClass: ???QZo??4??
		uid: ?	?H?2B,?g?e<?
		uid: ???M?Ip?WY?}
		mail: ?|????S
                            ?XiK?
		userPassword: \?%???\??#???R?~?c??????(<??Ys????Xz
                                                                      ??0#
		creatorsName: ?
                               $\D????[Qab?$GBW+k~??k??r?M
		modifiersName: ?
                                $\D????[Qab?$GBW+k~??k??r?M
		createTimestamp: ?.6?P???F?0?wa?
		modifyTimestamp: ?.6?P???F?0?wa?
		nsUniqueId: a?=??
		unhashed#user#password: ?$G?f?N?h???
==============================================================================================================



1-8. Run some modification ops.  E.g.,
   $ infadd -p <Master 1 port> -s "dc=example,dc=com" -u "cn=directory manager" -w <password>
   $ infadd -p <Master 2 port> -s "dc=test,dc=com" -u "cn=directory manager" -w <password>
==============================================================================================================
[root@snmaptest changelog]# infadd -p 30100 -s "dc=example,dc=com" -u "cn=directory manager" -w Secret123
Loading Given-Names ...
Loading Family-Names ...
infadd: 1 thread launched.

Rate:  153.00/thr ( 15.30/sec =65.3595ms/op), total: 153 (1 thr)
Rate:   68.00/thr (  6.80/sec =147.0588ms/op), total: 221 (1 thr)
Rate:   78.00/thr (  7.80/sec =128.2051ms/op), total: 299 (1 thr)
Rate:   83.00/thr (  8.30/sec =120.4819ms/op), total: 382 (1 thr)
Rate:   93.00/thr (  9.30/sec =107.5269ms/op), total: 475 (1 thr)
==============================================================================================================

2. Update changelog encryption along with the Certificate renewal
2-1. Export changelog db on Master servers.
   $ ldapmodify [...]
   dn: cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config
   changetype: modify
   add: nsds5Task
   nsds5Task: CL2LDIF
   $ ldapmodify [...]  
   dn: cn=replica,cn=dc\3Dtest\2Cdc\3Dcom,cn=mapping tree,cn=config
   changetype: modify
   add: nsds5Task
   nsds5Task: CL2LDIF

   Monitor the error log /var/log/dirsrv/slapd-master[12]/errors to check the export is successfully finished.
   [...] NSMMReplicationPlugin - Beginning changelog export of replica "5af4cd84-1dd211b2-b4b8f8dd-b6310000"
   [...] NSMMReplicationPlugin - Finished changelog export of replica "5af4cd84-1dd211b2-b4b8f8dd-b6310000"

===========================================================================================================
[root@snmaptest /]# ldapmodify -D "cn=directory manager" -w Secret123 -p 30100 -x -h localhost << EOF
dn: cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config
changetype: modify
add: nsds5Task
nsds5Task: CL2LDIF
EOF


[root@snmaptest changelog]# tail -f /var/log/dirsrv/slapd-M1/errors
[22/Sep/2011:14:49:50 +051800] - All database threads now stopped
[22/Sep/2011:14:49:50 +051800] - slapd stopped.
[22/Sep/2011:14:51:35 +051800] - 389-Directory/1.2.9.11 B2011.259.2023 starting up
[22/Sep/2011:14:51:35 +051800] attrcrypt - _back_crypt_cipher_init: No symmetric key found for cipher AES, attempting to create one...
[22/Sep/2011:14:51:36 +051800] - slapd started.  Listening on All Interfaces port 30100 for LDAP requests
[22/Sep/2011:14:51:36 +051800] - Listening on All Interfaces port 30101 for LDAPS requests
[22/Sep/2011:15:01:25 +051800] NSMMReplicationPlugin - Beginning total update of replica "agmt="cn=M1_to_M2" (snmaptest:30103)".
[22/Sep/2011:15:01:28 +051800] NSMMReplicationPlugin - Finished total update of replica "agmt="cn=M1_to_M2" (snmaptest:30103)". Sent 160 entries.
[22/Sep/2011:15:07:45 +051800] NSMMReplicationPlugin - Beginning total update of replica "agmt="cn=M1_to_M4" (snmaptest:30107)".
[22/Sep/2011:15:07:48 +051800] NSMMReplicationPlugin - Finished total update of replica "agmt="cn=M1_to_M4" (snmaptest:30107)". Sent 160 entries.
[22/Sep/2011:15:30:57 +051800] NSMMReplicationPlugin - Beginning changelog export of replica "6e06a173-e4fa11e0-9528d0a9-2fb8bf08"
[22/Sep/2011:15:30:58 +051800] NSMMReplicationPlugin - Finished changelog export of replica "6e06a173-e4fa11e0-9528d0a9-2fb8bf08"


===========================================================================================================



2-2. Check the exported changelog file on Master servers (one ldif file per changelog db).
   # ls /var/lib/dirsrv/slapd-master[12]/changelogdb/*.ldif
   /var/lib/dirsrv/slapd-master[12]/changelogdb/[...].ldif

   Changes in each changelog is base64 encoded.  E.g.,
   changetype: add
   replgen: 4d2b599c000000010000
   csn: 4d2b5a9c000000020000
   nsuniqueid: c63f9f01-1dd111b2-ad4a8494-47bd0000
   parentuniqueid: 40bbef65-1dd211b2-b4baf8dd-b6310000
   dn: uid=j0user0,ou=People,dc=test,dc=com
   change:: YWRkOiB1aWQKdWlkOiBqMHVzZXIwCi0KYWRkOiBnaXZlbk5hbWUKZ2l2ZW5OYW1lOiBq
    aWppMAotCmFkZDogb2JqZWN0Q2xhc3MKb2JqZWN0Q2xhc3M6IHRvcApvYmplY3RDbGFzczogcGVy
    c29uCm9iamVjdENsYXNzOiBvcmdhbml6YXRpb25hbFBlcnNvbgpvYmplY3RDbGFzczogaW5ldG9y
    Z3BlcnNvbgotCmFkZDogc24Kc246IHVzZXIwCi0KYWRkOiBjbgpjbjogamlqaTAgdXNlcjAKLQph
    ZGQ6IHVzZXJQYXNzd29yZAp1c2VyUGFzc3dvcmQ6IHtTU0hBfUI5K25pWTkzdE9XYzlhTHF5djdv
    b1MwVUJrdktlaUgzZndxNGFBPT0KLQphZGQ6IGNyZWF0b3JzTmFtZQpjcmVhdG9yc05hbWU6IGNu
    PWRpcmVjdG9yeSBtYW5hZ2VyCi0KYWRkOiBtb2RpZmllcnNOYW1lCm1vZGlmaWVyc05hbWU6IGNu
    PWRpcmVjdG9yeSBtYW5hZ2VyCi0KYWRkOiBjcmVhdGVUaW1lc3RhbXAKY3JlYXRlVGltZXN0YW1w
    OiAyMDExMDExMDE5MTQzNloKLQphZGQ6IG1vZGlmeVRpbWVzdGFtcAptb2RpZnlUaW1lc3RhbXA6
    IDIwMTEwMTEwMTkxNDM2WgotCmFkZDogbnNVbmlxdWVJZApuc1VuaXF1ZUlkOiBjNjNmOWYwMS0x
    ZGQxMTFiMi1hZDRhODQ5NC00N2JkMDAwMAotCmFkZDogdW5oYXNoZWQjdXNlciNwYXNzd29yZAp1
    bmhhc2hlZCN1c2VyI3Bhc3N3b3JkOiBqMHVzZXIwCi0K
   Decode the change and make sure it is not encrypted:
   add: uid
   uid: juser0
   -
   add: givenName
   givenName: jiji
   -
   add: objectClass
   [...]
   -
   add: nsUniqueId
   nsUniqueId: af00b181-1dd111b2-b4bbf8dd-b6310000
   -
   add: unhashed#user#password
   unhashed#user#password: juser0
===========================================================
[root@snmaptest changelog]# ls /var/lib/dirsrv/slapd-M1/db/changelog/
6e06a173-e4fa11e0-9528d0a9-2fb8bf08_4e7afb14000000010000.db4  6e06a173-e4fa11e0-9528d0a9-2fb8bf08.ldif  6e06a173-e4fa11e0-9528d0a9-2fb8bf08.sema  DBVERSION

ldif file contents
changetype: add
replgen: 4e7afb14000000010000
csn: 4e7b0243000000010000
nsuniqueid: a4461f82-e4fe11e0-9528d0a9-2fb8bf08
parentuniqueid: 6cd57425-e4fa11e0-9528d0a9-2fb8bf08
dn: uid=amsharma2,ou=people,dc=example,dc=com
change:: YWRkOiBjbgpjbjogYW1zCi0KYWRkOiBzbgpzbjogYW1zCi0KYWRkOiBnaXZlbk5hbWUKZ
 2l2ZW5OYW1lOiBhbXMKLQphZGQ6IG9iamVjdENsYXNzCm9iamVjdENsYXNzOiB0b3AKb2JqZWN0Q2
 xhc3M6IHBlcnNvbgpvYmplY3RDbGFzczogb3JnYW5pemF0aW9uYWxQZXJzb24Kb2JqZWN0Q2xhc3M
 6IGluZXRPcmdQZXJzb24KLQphZGQ6IHVpZAp1aWQ6IGFtcwp1aWQ6IGFtc2hhcm1hMgotCmFkZDog
 bWFpbAptYWlsOiBhbXNAZXhhbXBsZS5jb20KLQphZGQ6IHVzZXJQYXNzd29yZAp1c2VyUGFzc3dvc
 mQ6OiBlMU5UU0VGOVF6bE9hMVpDZDNWa2EzRXJhVU0zT0VkTVVGVk5OM2MyTWtRMVdsbHVMMFZJYU
 ZocVMxRTlQUT0KID0KLQphZGQ6IGNyZWF0b3JzTmFtZQpjcmVhdG9yc05hbWU6IGNuPWRpcmVjdG9
 yeSBtYW5hZ2VyCi0KYWRkOiBtb2RpZmllcnNOYW1lCm1vZGlmaWVyc05hbWU6IGNuPWRpcmVjdG9y
 eSBtYW5hZ2VyCi0KYWRkOiBjcmVhdGVUaW1lc3RhbXAKY3JlYXRlVGltZXN0YW1wOiAyMDExMDkyM
 jA5MzkxNVoKLQphZGQ6IG1vZGlmeVRpbWVzdGFtcAptb2RpZnlUaW1lc3RhbXA6IDIwMTEwOTIyMD
 kzOTE1WgotCmFkZDogbnNVbmlxdWVJZApuc1VuaXF1ZUlkOiBhNDQ2MWY4Mi1lNGZlMTFlMC05NTI
 4ZDBhOS0yZmI4YmYwOAotCmFkZDogdW5oYXNoZWQjdXNlciNwYXNzd29yZAp1bmhhc2hlZCN1c2Vy
 I3Bhc3N3b3JkOiBhbXNhbXNhbXMKLQo=

Decoded contents
[root@snmaptest changelog]# base64 -d test.b64
add: cn
cn: ams
-
add: sn
sn: ams
-
add: givenName
===========================================================

2-3. Recommend to back up DBs on each server
   # Stop the servers
   # /usr/lib[64]/dirsrv/slapd-ID/db2bak
2-4. Stop the server and disable changelog encryption.
   Remove these config entries from dse.ldif (2 entries per backend -- suffix):
     dn: cn=3DES,cn=encrypted attribute keys,cn=<backend>,cn=ldbm database,cn=plugins,cn=config
     dn: cn=AES,cn=encrypted attribute keys,cn=<backend>,cn=ldbm database,cn=plugins,cn=config
   Remove these config attr values from cn=changelog5,cn=config
     nsslapd-encryptionalgorithm: AES
     nsSymmetricKey:: LrKrvjtihBJA8G5aBohkABd2pUyM7iwn2EO1Y7QpU7iJhHDsfV+j12prQBp3
      [...]
===========================================================
[root@snmaptest changelog]# service dirsrv stop
Shutting down dirsrv: 
    M1...                                                  [  OK  ]
    M2...                                                  [  OK  ]
    M3...                                                  [  OK  ]
    M4...                                                  [  OK  ]
    snmaptest...                                           [  OK  ]
[root@snmaptest changelog]# 

Removed specified enteries.
=============================================================


2-5. Renew the server certificate
2-6. Stop Master servers and set nsslapd-encryptionalgorithm.  The allowed value is AES or 3DES.
   dn: cn=changelog5,cn=config
   [...]
   nsslapd-encryptionalgorithm: AES
2-7. Restart the servers and import the changelog
   $ ldapmodify [...]
   dn: cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config
   changetype: modify
   add: nsds5Task
   nsds5Task: LDIF2CL
   $ ldapmodify [...]
   dn: cn=replica,cn=dc\3Dtest\2Cdc\3Dcom,cn=mapping tree,cn=config
   changetype: modify
   add: nsds5Task
   nsds5Task: LDIF2CL

======================================================================================
[root@snmaptest fourwaymmr]# service dirsrv stop
Shutting down dirsrv: 
    M1...                                                  [  OK  ]
    M2...                                                  [  OK  ]
    M3...                                                  [  OK  ]
    M4...                                                  [  OK  ]
    snmaptest...                                           [  OK  ]
[root@snmaptest fourwaymmr]# vim /etc/dirsrv/slapd-M1/dse.ldif
[root@snmaptest fourwaymmr]# service dirsrv start
Starting dirsrv: 
    M1...                                                  [  OK  ]
    M2...                                                  [  OK  ]
    M3...                                                  [  OK  ]
    M4...                                                  [  OK  ]
    snmaptest...                                           [  OK  ]
[root@snmaptest fourwaymmr]# ldapmodify -D "cn=directory manager" -w Secret123 -p 30100 -x -h localhost << EOF
> dn: cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config
> changetype: modify
> add: nsds5Task
> nsds5Task: LDIF2CL
> EOF
modifying entry "cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config"
[root@snmaptest fourwaymmr]# tail -f /var/log/dirsrv/slapd-M1
tail: error reading `/var/log/dirsrv/slapd-M1': Is a directory
tail: /var/log/dirsrv/slapd-M1: cannot follow end of this type of file; giving up on this name
[root@snmaptest fourwaymmr]# tail -f /var/log/dirsrv/slapd-M1/errors
[22/Sep/2011:16:57:23 +051800] - slapd shutting down - closing down internal subsystems and plugins
[22/Sep/2011:16:57:24 +051800] - Waiting for 4 database threads to stop
[22/Sep/2011:16:57:24 +051800] - All database threads now stopped
[22/Sep/2011:16:57:24 +051800] - slapd stopped.
[22/Sep/2011:16:58:06 +051800] - 389-Directory/1.2.9.11 B2011.259.2023 starting up
[22/Sep/2011:16:58:06 +051800] attrcrypt - _back_crypt_cipher_init: No symmetric key found for cipher AES, attempting to create one...
[22/Sep/2011:16:58:06 +051800] - slapd started.  Listening on All Interfaces port 30100 for LDAP requests
[22/Sep/2011:16:58:06 +051800] - Listening on All Interfaces port 30101 for LDAPS requests
[22/Sep/2011:16:59:15 +051800] NSMMReplicationPlugin - Beginning changelog import of replica "6e06a173-e4fa11e0-9528d0a9-2fb8bf08"
[22/Sep/2011:16:59:26 +051800] NSMMReplicationPlugin - Finished changelog import of replica "6e06a173-e4fa11e0-9528d0a9-2fb8bf08"

======================================================================================
   Monitor the error log /var/log/dirsrv/slapd-master[12]/errors to check the import is successfully finished.
   [...] NSMMReplicationPlugin - Beginning changelog import of replica "5af4cd82-1dd211b2-b4b8f8dd-b6310000"
   [...] NSMMReplicationPlugin - Finished changelog import of replica "5af4cd82-1dd211b2-b4b8f8dd-b6310000"
2-8. For testing, modify something on the both masters and check the change is replicated to the replicas.

Hence Verified.


Note You need to log in before you can comment on or make changes to this bug.