Bug 663752
| Summary: | Cert renewal for attrcrypt and encchangelog | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | [Retired] 389 | Reporter: | Noriko Hosoi <nhosoi> | ||||||||||
| Component: | Security - SSL | Assignee: | Noriko Hosoi <nhosoi> | ||||||||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Viktor Ashirov <vashirov> | ||||||||||
| Severity: | medium | Docs Contact: | |||||||||||
| Priority: | high | ||||||||||||
| Version: | 1.2.7 | CC: | amsharma, jgalipea, rmeggins | ||||||||||
| Target Milestone: | --- | ||||||||||||
| Target Release: | --- | ||||||||||||
| Hardware: | Unspecified | ||||||||||||
| OS: | Unspecified | ||||||||||||
| Whiteboard: | |||||||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||||||
| Doc Text: | Story Points: | --- | |||||||||||
| Clone Of: | |||||||||||||
| : | 713317 713318 (view as bug list) | Environment: | |||||||||||
| Last Closed: | 2015-12-07 16:44:56 UTC | Type: | --- | ||||||||||
| Regression: | --- | Mount Type: | --- | ||||||||||
| Documentation: | --- | CRM: | |||||||||||
| Verified Versions: | Category: | --- | |||||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||||
| Embargoed: | |||||||||||||
| Bug Depends On: | |||||||||||||
| Bug Blocks: | 434915, 708096, 713317, 713318 | ||||||||||||
| Attachments: |
|
||||||||||||
We already have a replication task to export the changelog:
$ ldapmodify ...
dn: cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config
changetype: modify
add: nsds5Task
nsds5Task: CL2LDIF
It dumps the contents of changelog into the ldif file.
# pwd
/var/lib/dirsrv/slapd-ID/changelogdb
# ls
b50f2982-1dd111b2-9c21b4e8-fd670000_4d13a124000000010000.db4 b50f2982-1dd111b2-9c21b4e8-fd670000.sema
b50f2982-1dd111b2-9c21b4e8-fd670000.ldif
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Snippet of the ldif:
changetype: add
replgen: 4d13a124000000010000
csn: 4d13a1b8000000020000
nsuniqueid: 1cc57481-1dd211b2-91a3b842-4b530000
parentuniqueid: b2133983-1dd111b2-825ea6cb-4a0f0000
dn: uid=s1user0,ou=People,dc=example,dc=com
change:: YWRkOiB1aWQKdWlkOiBzMXVzZXIwCi0KYWRkOiBnaXZlbk5hbWUKZ2l2ZW5OYW1lOiBz
MQotCmFkZDogb2JqZWN0Q2xhc3MKb2JqZWN0Q2xhc3M6IHRvcApvYmplY3RDbGFzczogcGVyc29u
Cm9iamVjdENsYXNzOiBvcmdhbml6YXRpb25hbFBlcnNvbgpvYmplY3RDbGFzczogaW5ldG9yZ3Bl
cnNvbgotCmFkZDogc24Kc246IHVzZXIwCi0KYWRkOiBjbgpjbjogczEgdXNlcjAKLQphZGQ6IHVz
ZXJQYXNzd29yZAp1c2VyUGFzc3dvcmQ6IHtTU0hBfS9oQnY0N2dYSk1wZlZEb2E1bXZKaUxpQk1p
cFZWTjdHVEtldmR3PT0KLQphZGQ6IGNyZWF0b3JzTmFtZQpjcmVhdG9yc05hbWU6IHVpZD1hZG1p
bixvdT1hZG1pbmlzdHJhdG9ycyxvdT10b3BvbG9neW1hbmFnZW1lbnQsbz1uZXRzY2FwZXJvb3QK
LQphZGQ6IG1vZGlmaWVyc05hbWUKbW9kaWZpZXJzTmFtZTogdWlkPWFkbWluLG91PWFkbWluaXN0
cmF0b3JzLG91PXRvcG9sb2d5bWFuYWdlbWVudCxvPW5ldHNjYXBlcm9vCiB0Ci0KYWRkOiBjcmVh
dGVUaW1lc3RhbXAKY3JlYXRlVGltZXN0YW1wOiAyMDEwMTIyMzE5MjMzNVoKLQphZGQ6IG1vZGlm
eVRpbWVzdGFtcAptb2RpZnlUaW1lc3RhbXA6IDIwMTAxMjIzMTkyMzM1WgotCmFkZDogbnNVbmlx
dWVJZApuc1VuaXF1ZUlkOiAxY2M1NzQ4MS0xZGQyMTFiMi05MWEzYjg0Mi00YjUzMDAwMAotCmFk
ZDogdW5oYXNoZWQjdXNlciNwYXNzd29yZAp1bmhhc2hlZCN1c2VyI3Bhc3N3b3JkOiBzMXVzZXIw
Ci0K
Base64 decoded result:
add: uid
uid: s1user0
-
add: givenName
givenName: s1
-
add: objectClass
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetorgperson
-
add: sn
sn: user0
-
add: cn
cn: s1 user0
-
add: userPassword
userPassword: {SSHA}/hBv47gXJMpfVDoa5mvJiLiBMipVVN7GTKevdw==
-
add: creatorsName
creatorsName: uid=admin,ou=administrators,ou=topologymanagement,o=netscaperoot
-
add: modifiersName
modifiersName: uid=admin,ou=administrators,ou=topologymanagement,o=netscaperoo
t
-
add: createTimestamp
createTimestamp: 20101223192335Z
-
add: modifyTimestamp
modifyTimestamp: 20101223192335Z
-
add: nsUniqueId
nsUniqueId: 1cc57481-1dd211b2-91a3b842-4b530000
-
add: unhashed#user#password
unhashed#user#password: s1user0
-
Note: The original changelog is encrypted with AES. The exported ldif is not.
Created attachment 472939 [details] git patch file (master) Description: In fixing Bug 182507, the feature to encrypt changelogs had been introduced. The changelog encryption depends on the server certificate as the attrcrypt does. When the server certificate is renewed, the encrypted changelog won't be decrypted. This patch implements/completes the feature to export and import the contents of the changelog. See also this section for the steps to export/import changelogs. http://directory.fedoraproject.org/wiki/Changelog_Encryption#Steps_for_Certificate_Renewal Reviewed by Nathan (Thank you!!!) Pushed to master. $ git merge 663752 Updating 66a666c..ad3c528 Fast-forward ldap/servers/plugins/replication/cl5_api.c | 277 +++++++++++++--- ldap/servers/plugins/replication/cl5_clcache.c | 15 +- .../plugins/replication/repl5_replica_config.c | 345 +++++++++++++++++++- ldap/servers/plugins/replication/repl5_ruv.c | 7 +- ldap/servers/slapd/slapi-private.h | 1 + 5 files changed, 576 insertions(+), 69 deletions(-) $ git push Counting objects: 23, done. Delta compression using up to 2 threads. Compressing objects: 100% (12/12), done. Writing objects: 100% (12/12), 5.77 KiB, done. Total 12 (delta 10), reused 0 (delta 0) To ssh://git.fedorahosted.org/git/389/ds.git 66a666c..ad3c528 master -> master Following http://directory.fedoraproject.org/wiki/Changelog_Encryption#Steps_for_Certificate_Renewal ============================= This is part one tested OK ============================ 1. Preparation 1-1. Install 3 servers with at least 2 backend databases to replicate. e.g., suffix "dc=example,dc=com", "dc=test,dc=com" 1-2. Setup SSL on Master servers. 1-3. Setup Master 1 <--> Master 2 | v Read only replica 20100 <--->20102 1-4. Stop Master servers and set nsslapd-encryptionalgorithm. The allowed value is AES or 3DES. dn: cn=changelog5,cn=config [...] nsslapd-encryptionalgorithm: AES [root@rhel61-ds90-amita ~]# /usr/lib64/dirsrv/slapd-M1/stop-slapd [root@rhel61-ds90-amita ~]# service dirsrv status dirsrv M1 is stopped dirsrv M2 (pid 1342) is running... objectClass: top objectClass: extensibleobject cn: changelog5 nsslapd-changelogdir: /var/lib/dirsrv/slapd-M1/db/changelog nsslapd-encryptionalgorithm: AES creatorsName: cn=directory manager modifiersName: cn=directory manager createTimestamp: 20110607111531Z modifyTimestamp: 20110607111531Z 1-5. Restart Master servers, and initialialize replicas on each agreement on Master 1. /usr/lib64/dirsrv/slapd-M1/start-slapd 1-6. Verify the replication topology is correctly set up by adding at least one entry to each backend on Master servers. [root@rhel61-ds90-amita ~]# ldapadd -x -h localhost -p 20100 -D "cn=Directory Manager" -w Secret123 << EOF dn: uid=amsharma2,ou=people,dc=replsuffix,dc=com cn: ams sn: ams givenname: ams objectclass: top objectclass: person objectclass: organizationalPerson objectclass: inetOrgPerson uid: ams mail: ams userpassword: amsamsams EOF adding new entry "uid=amsharma2,ou=people,dc=replsuffix,dc=com" [root@rhel61-ds90-amita ~]# ldapsearch -x -p 20102 -h localhost -D "cn=Directory Manager" -w Secret123 -b "dc=replsuffix,dc=com" | grep amsharma2 # amsharma2, People, replsuffix.com dn: uid=amsharma2,ou=People,dc=replsuffix,dc=com uid: amsharma2 1-7. Dump Master servers' changelog to confirm the changelogs are encrypted. One changelog per replica; If you set up replicas on 2 backends, e.g., dc=example,dc=com and dc=test,dc=com, there are 2 changelog db files. # dbscan -f /var/lib/dirsrv/slapd-master[12]/changelogdb/[...].db4 dbscan -f bb036a82-90f711e0-a47e8da0-ec4827e4_4dee08e2000000010000.db dbid: 4def39fb000000010000 replgen: 1307523578 Wed Jun 8 14:29:38 2011 csn: 4def39fb000000010000 uniqueid: 8651ea81-91ad11e0-a47e8da0-ec4827e4 parentuniqueid: 5b0cc403-90f711e0-9a609c91-c5a5a6e6 dn: uid=amsharma2,ou=people,dc=replsuffix,dc=com operation: add cn: g????Y?|f?}?:;U sn: g????Y?|f?}?:;U givenName: g????Y?|f?}?:;U objectClass: ??v?G[q>&U objectClass: a?n??z?z???} objectClass: ??[?F?L?@?Q????C?? ?1?? ?? objectClass: ??2?/9?q???? uid: g????Y?|f?}?:;U ?Ga?? uid: ?b2??? mail: ?e????~?vi? c??(?GX??=QF??o?l{erPassword: ??n?s?%\??b ???cYE?Z{?82??0pn?{?8{rsName: ?{?F?=k ???cYE?Z{?82??0pn?{?8{ersName: ?{?F?=k createTimestamp: X??V?F???2?? modifyTimestamp: X??V?F???2?? ??J?|????0 nsUniqueId: ??e-X?]??XH???r)??dn ?d.Yq? ???B unhashed#user#password: c?*?[8T%sV9? 1-8. Run some modification ops. E.g., $ infadd -p <Master 1 port> -s "dc=example,dc=com" -u "cn=directory manager" -w <password> $ infadd -p <Master 2 port> -s "dc=test,dc=com" -u "cn=directory manager" -w <password> ===================================================== This is part two but I am facing the server crash here - ======================================================== [root@rhel61-ds90-amita changelog]# ldapmodify -D "cn=directory manager" -w Secret123 -p 20100 -x -h localhost << EOF > dn: cn=replica,cn=dc\3Dreplsuffix\2Cdc\3Dcom,cn=mapping tree,cn=config > changetype: modify > add: nsds5Task > nsds5Task: CL2LDIF > EOF modifying entry "cn=replica,cn=dc\3Dreplsuffix\2Cdc\3Dcom,cn=mapping tree,cn=config" ldap_result: Can't contact LDAP server (-1) [root@rhel61-ds90-amita changelog]# ldapmodify -x -h localhost -p 20100 -D "cn=directory manager" -w Secret123 << EOF > dn: cn=replica,cn=dc\3Dreplsuffix\2Cdc\3Dcom,cn=mapping tree,cn=config > changetype: modify > add: nsds5Task > nsds5Task: CL2LDIF > EOF ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1) [root@rhel61-ds90-amita changelog]# service dirsrv status dirsrv M1 dead but pid file exists dirsrv M2 (pid 1342) is running... dirsrv M3 (pid 1422) is running... dirsrv M4 (pid 1507) is running... dirsrv rhel61-ds90-amita (pid 1580) is running... [root@rhel61-ds90-amita changelog]# /usr/lib64/dirsrv/slapd-M1/stop-slapd Server not running [root@rhel61-ds90-amita changelog]# /usr/lib64/dirsrv/slapd-M1/start-slapd [root@rhel61-ds90-amita changelog]# service dirsrv status dirsrv M1 (pid 1973) is running... dirsrv M2 (pid 1342) is running... dirsrv M3 (pid 1422) is running... dirsrv M4 (pid 1507) is running... dirsrv rhel61-ds90-amita (pid 1580) is running... [root@rhel61-ds90-amita changelog]# ldapmodify -x -h localhost -p 20100 -D "cn=directory manager" -w Secret123 << EOF > dn: cn=replica,cn=dc\3Dreplsuffix\2Cdc\3Dcom,cn=mapping tree,cn=config > changetype: modify > add: nsds5Task > nsds5Task: CL2LDIF > EOF modifying entry "cn=replica,cn=dc\3Dreplsuffix\2Cdc\3Dcom,cn=mapping tree,cn=config" ldap_result: Can't contact LDAP server (-1) [root@rhel61-ds90-amita changelog]# service dirsrv status dirsrv M1 dead but pid file exists dirsrv M2 (pid 1342) is running... dirsrv M3 (pid 1422) is running... dirsrv M4 (pid 1507) is running... dirsrv rhel61-ds90-amita (pid 1580) is running... [root@rhel61-ds90-amita changelog]# ======= LOGS ========= [root@rhel61-ds90-amita changelog]# tail -f /var/log/dirsrv/slapd-M1/errors [08/Jun/2011:14:39:57 +051800] NSMMReplicationPlugin - changelog program - _cl5ReadMod: decrypting "createTimestamp: 32" failed [08/Jun/2011:14:39:57 +051800] attrcrypt - _back_crypt_crypto_op failed on cipher AES : -8188 - security library has experienced an input length error. [08/Jun/2011:14:39:57 +051800] attrcrypt - _back_crypt_crypto_op failed on cipher AES : -8188 - security library has experienced an input length error. [08/Jun/2011:14:39:57 +051800] NSMMReplicationPlugin - changelog program - _cl5ReadMod: decrypting "modifyTimestamp: 32" failed [08/Jun/2011:14:39:57 +051800] attrcrypt - _back_crypt_crypto_op failed on cipher AES : -8188 - security library has experienced an input length error. [08/Jun/2011:14:39:57 +051800] attrcrypt - _back_crypt_crypto_op failed on cipher AES : -8188 - security library has experienced an input length error. [08/Jun/2011:14:39:57 +051800] NSMMReplicationPlugin - changelog program - _cl5ReadMod: decrypting "nsUniqueId: 65" failed [08/Jun/2011:14:39:57 +051800] attrcrypt - _back_crypt_crypto_op failed on cipher AES : -8188 - security library has experienced an input length error. [08/Jun/2011:14:39:57 +051800] attrcrypt - _back_crypt_crypto_op failed on cipher AES : -8188 - security library has experienced an input length error. [08/Jun/2011:14:39:57 +051800] NSMMReplicationPlugin - changelog program - _cl5ReadMod: decrypting "unhashed#user#password: 61" failed [root@rhel61-ds90-amita changelog]# tail -f /var/log/dirsrv/slapd-M1/access [08/Jun/2011:14:36:03 +051800] conn=7 op=15 EXT oid="2.16.840.1.113730.3.5.12" [08/Jun/2011:14:36:03 +051800] conn=7 op=15 RESULT err=0 tag=120 nentries=0 etime=0 [08/Jun/2011:14:36:03 +051800] conn=7 op=16 EXT oid="2.16.840.1.113730.3.5.5" name="Netscape Replication End Session" [08/Jun/2011:14:36:03 +051800] conn=7 op=16 RESULT err=0 tag=120 nentries=0 etime=0 [08/Jun/2011:14:37:00 +051800] conn=9 op=16 UNBIND [08/Jun/2011:14:37:00 +051800] conn=9 op=16 fd=66 closed - U1 [08/Jun/2011:14:37:02 +051800] conn=8 op=18 UNBIND [08/Jun/2011:14:37:02 +051800] conn=8 op=18 fd=65 closed - U1 [08/Jun/2011:14:37:04 +051800] conn=7 op=17 UNBIND [08/Jun/2011:14:37:04 +051800] conn=7 op=17 fd=64 closed - U1 Please guide. -Amita Amita, Thanks for the good test case. I could use your VM 10.16.98.212 to investigate your crash problem. Let me confirm one thing. On M1, have you renewed the certificate? The CL2LDIF task shows lots of errors like this. It looks decrypting the changelogs are all failing on M1 and not sure what's happening... [..] attrcrypt - _back_crypt_crypto_op failed on cipher AES : -8188 - security library has experienced an input length error. I've installed my local build. I had to wipe out the existing changelog and add some changes to the server. Now, I can successfully export the changelog like this. # ldapmodify -x -h localhost -p 20100 -D "cn=directory manager" -w Secret123 dn: cn=replica,cn=dc\3Dreplsuffix\2Cdc\3Dcom,cn=mapping tree,cn=config changetype: modify add: nsds5Task nsds5Task: CL2LDIF modifying entry "cn=replica,cn=dc\3Dreplsuffix\2Cdc\3Dcom,cn=mapping tree,cn=config" Could you repeat what you did to reproduce the crash and see how it goes with my local build? Since there's a crash issue, I'm changing the status to ASSIGNED. Created attachment 504780 [details]
git patch file (master)
Description: When changelog is encrypted and the certificate
used for the encryption has a problem (e.g., expired, renewed,
etc.), running the CL2LDIF task could crash the server. This
patch is adding more error checks for the decrypted result.
If a problem is found, it skips the change.
Note: this problem was found by Amita's verification effort.
Thanks, Amita!
Reviewed by Rich (Thank you!!) Pushed to master. $ git merge 663752 Updating 9cd0752..c35e240 Fast-forward ldap/servers/plugins/replication/cl5_api.c | 36 +++++++++++++++++++++------ ldap/servers/slapd/util.c | 20 +++++++++++---- 2 files changed, 42 insertions(+), 14 deletions(-) $ git push Counting objects: 17, done. Delta compression using up to 4 threads. Compressing objects: 100% (9/9), done. Writing objects: 100% (9/9), 1.31 KiB, done. Total 9 (delta 7), reused 0 (delta 0) To ssh://git.fedorahosted.org/git/389/ds.git 9cd0752..c35e240 master -> master pushed to RHEL-6: (cherry picked from commit c35e240ff8a65225b8e1f890ccdc54da7533dbcf) $ git cherry-pick c35e240ff8a65225b8e1f890ccdc54da7533dbcf [RHEL-6 cad7963] Bug 663752 - Cert renewal for attrcrypt and encchangelog 2 files changed, 42 insertions(+), 14 deletions(-) $ git push redhat RHEL-6 Counting objects: 17, done. Delta compression using up to 4 threads. Compressing objects: 100% (9/9), done. Writing objects: 100% (9/9), 1.36 KiB, done. Total 9 (delta 7), reused 0 (delta 0) To ssh://git.engineering.redhat.com/srv/git/users/rmeggins/ds.git 4312b7a..da4c064 RHEL-6 -> RHEL-6 pushed to ds-replication-RHEL-6, as well. (cherry picked from commit c35e240ff8a65225b8e1f890ccdc54da7533dbcf) $ git cherry-pick c35e240ff8a65225b8e1f890ccdc54da7533dbcf [ds-replication-RHEL-6 5984448] Bug 663752 - Cert renewal for attrcrypt and encchangelog 2 files changed, 42 insertions(+), 14 deletions(-) $ git push redhat ds-replication-RHEL-6 Counting objects: 17, done. Delta compression using up to 4 threads. Compressing objects: 100% (9/9), done. Writing objects: 100% (9/9), 1.36 KiB, done. Total 9 (delta 7), reused 0 (delta 0) To ssh://git.engineering.redhat.com/srv/git/users/rmeggins/ds.git 80f966a..9ecc6ff ds-replication-RHEL-6 -> ds-replication-RHEL-6 Created attachment 505132 [details]
git patch file (master)
Description: There was a bug in using openldap API ldif_read_record
in cl5ImportLDIF (replication/cl5_api.c). The API ldif_read_record
reuses the buffer allocated internally, where buffer length variable
plays a role to determine the current buffer is large enough for the
new ldif line. The caller function cl5ImportLDIF freed the buffer
without setting 0 to the length. It caused segfault.
This patch sets 0 to the buffer length when the buffer is freed.
Reviewed by Rich (Thank you!) Pushed to master. $ git merge 663752 Updating c35e240..26695d3 Fast-forward ldap/servers/plugins/replication/cl5_api.c | 13 +++++++++---- 1 files changed, 9 insertions(+), 4 deletions(-) $ git push Counting objects: 13, done. Delta compression using up to 4 threads. Compressing objects: 100% (7/7), done. Writing objects: 100% (7/7), 957 bytes, done. Total 7 (delta 5), reused 0 (delta 0) To ssh://git.fedorahosted.org/git/389/ds.git c35e240..26695d3 master -> master ds-replication-RHEL-6: (cherry picked from commit 26695d3fd9cab800b141f67353e400ae65ed732e) $ git cherry-pick 26695d3fd9cab800b141f67353e400ae65ed732e [ds-replication-RHEL-6 fbe5f6c] Bug 663752 - Cert renewal for attrcrypt and encchangelog 1 files changed, 9 insertions(+), 4 deletions(-) $ git push redhat ds-replication-RHEL-6 Counting objects: 13, done. Delta compression using up to 4 threads. Compressing objects: 100% (7/7), done. Writing objects: 100% (7/7), 999 bytes, done. Total 7 (delta 5), reused 0 (delta 0) To ssh://git.engineering.redhat.com/srv/git/users/rmeggins/ds.git 9ecc6ff..275ab83 ds-replication-RHEL-6 -> ds-replication-RHEL-6 Created attachment 515807 [details]
git patch file (master)
Description: Replica config modify callback replica_config_post_
modify (repl5_replica_config.c) calls an internal modify API with
s_configLock held. The modify ends up calling a replica config
callback, in which it tries to acquire the same s_configLock and
it hangs there since the locking function PR_Lock is not re-entrant.
This patch avoids calling the internal modify API inside of s_configLock.
Reviewed by Rich (Thanks!!). Pushed to master. $ git am `pwd`/0001-Bug-663752-Cert-renewal-for-attrcrypt-and-encchangel.patch Applying: Bug 663752 - Cert renewal for attrcrypt and encchangelog /export/src/ds90/ldapserver/ldapserver/.git/rebase-apply/patch:63: trailing whitespace. $ git push Counting objects: 13, done. Delta compression using up to 2 threads. Compressing objects: 100% (7/7), done. Writing objects: 100% (7/7), 1.16 KiB, done. Total 7 (delta 5), reused 0 (delta 0) To ssh://git.fedorahosted.org/git/389/ds.git 7a0548b..a5fdcdd master -> master Pushed to ds-replication-RHEL-6, as well. (cherry picked from commit a5fdcddd4acc0811faa4a0152218a9702c979115) $ git push redhat ds-replication-RHEL-6 nhosoi.redhat.com's password: Counting objects: 13, done. Delta compression using up to 2 threads. Compressing objects: 100% (7/7), done. Writing objects: 100% (7/7), 1.20 KiB, done. Total 7 (delta 5), reused 0 (delta 0) To ssh://git.engineering.redhat.com/srv/git/users/rmeggins/ds.git adb1753..fae50d6 ds-replication-RHEL-6 -> ds-replication-RHEL-6 Note: this fix is in ds-replication-1.2.8.7 or newer. 1. Preparation
1-1. Install 3 servers with at least 2 backend databases to replicate.
e.g., suffix "dc=example,dc=com", "dc=test,dc=com"
1-2. Setup SSL on Master servers.
1-3. Setup Master 1 <--> Master 2
|
v
Read only replica
1-4. Stop Master servers and set nsslapd-encryptionalgorithm. The allowed value is AES or 3DES.
dn: cn=changelog5,cn=config
[...]
nsslapd-encryptionalgorithm: AES
=================================================================
[root@snmaptest /]# /usr/lib64/dirsrv/slapd-M1/stop-slapd
[root@snmaptest /]# service dirsrv status
dirsrv M1 is stopped
dirsrv M2 (pid 15253) is running...
dn: cn=changelog5,cn=config
objectClass: top
objectClass: extensibleobject
cn: changelog5
nsslapd-changelogdir: /var/lib/dirsrv/slapd-M1/db/changelog
creatorsName: cn=directory manager
modifiersName: cn=directory manager
createTimestamp: 20110922090836Z
modifyTimestamp: 20110922090836Z
nsslapd-encryptionalgorithm: AES
=================================================================
1-5. Restart Master servers, and initialialize replicas on each agreement on Master 1.
=======================================================================================
[root@snmaptest ~]# ldapmodify -x -h localhost -p 30100 -D "cn=Directory Manager" -w Secret123 << EOF
> dn: cn=M1_to_M2,cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config
> changetype: modify
> replace: nsds5beginreplicarefresh
> nsds5beginreplicarefresh: start
> EOF
modifying entry "cn=M1_to_M2,cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config"
[root@snmaptest ~]# ldapmodify -x -h localhost -p 30100 -D "cn=Directory Manager" -w Secret123 << EOF
dn: cn=M1_to_M4,cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config
changetype: modify
replace: nsds5beginreplicarefresh
nsds5beginreplicarefresh: start
EOF
modifying entry "cn=M1_to_M4,cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config"
=======================================================================================
1-6. Verify the replication topology is correctly set up by adding at least one entry to each backend on Master servers.
==============================================================================================================
[root@snmaptest ~]# ldapadd -x -h localhost -p 30100 -D "cn=Directory Manager" -w Secret123 << EOF
> dn: uid=amsharma2,ou=people,dc=example,dc=com
> cn: ams
> sn: ams
> givenname: ams
> objectclass: top
> objectclass: person
> objectclass: organizationalPerson
> objectclass: inetOrgPerson
> uid: ams
> mail: ams
> userpassword: amsamsams
> EOF
adding new entry "uid=amsharma2,ou=people,dc=example,dc=com"
[root@snmaptest ~]# ldapsearch -x -p 30102 -h localhost -D "cn=Directory Manager" -w Secret123 -b "dc=example,dc=com" | grep amsharma2
# amsharma2, People, example.com
dn: uid=amsharma2,ou=People,dc=example,dc=com
uid: amsharma2
==============================================================================================================
1-7. Dump Master servers' changelog to confirm the changelogs are encrypted.
One changelog per replica;
If you set up replicas on 2 backends, e.g., dc=example,dc=com and dc=test,dc=com, there are 2 changelog db files.
# dbscan -f /var/lib/dirsrv/slapd-master[12]/changelogdb/[...].db4
==============================================================================================================
[root@snmaptest changelog]# dbscan -f 6e06a173-e4fa11e0-9528d0a9-2fb8bf08_4e7afb14000000010000.db4
dbid: 4e7b0243000000010000
replgen: 1316684355 Thu Sep 22 15:09:15 2011
csn: 4e7b0243000000010000
uniqueid: a4461f82-e4fe11e0-9528d0a9-2fb8bf08
parentuniqueid: 6cd57425-e4fa11e0-9528d0a9-2fb8bf08
dn: uid=amsharma2,ou=people,dc=example,dc=com
operation: add
cn: ? ?H?2B,?g?e<?
sn: ? ?H?2B,?g?e<?
givenName: ? ?H?2B,?g?e<?
objectClass:
?u???nB1??4
?Z??Ww_0g\ objectClass: i?]
4?,??;?.e?? objectClass: ??)?b?H)-??\4P
objectClass: ???QZo??4??
uid: ? ?H?2B,?g?e<?
uid: ???M?Ip?WY?}
mail: ?|????S
?XiK?
userPassword: \?%???\??#???R?~?c??????(<??Ys????Xz
??0#
creatorsName: ?
$\D????[Qab?$GBW+k~??k??r?M
modifiersName: ?
$\D????[Qab?$GBW+k~??k??r?M
createTimestamp: ?.6?P???F?0?wa?
modifyTimestamp: ?.6?P???F?0?wa?
nsUniqueId: a?=??
unhashed#user#password: ?$G?f?N?h???
==============================================================================================================
1-8. Run some modification ops. E.g.,
$ infadd -p <Master 1 port> -s "dc=example,dc=com" -u "cn=directory manager" -w <password>
$ infadd -p <Master 2 port> -s "dc=test,dc=com" -u "cn=directory manager" -w <password>
==============================================================================================================
[root@snmaptest changelog]# infadd -p 30100 -s "dc=example,dc=com" -u "cn=directory manager" -w Secret123
Loading Given-Names ...
Loading Family-Names ...
infadd: 1 thread launched.
Rate: 153.00/thr ( 15.30/sec =65.3595ms/op), total: 153 (1 thr)
Rate: 68.00/thr ( 6.80/sec =147.0588ms/op), total: 221 (1 thr)
Rate: 78.00/thr ( 7.80/sec =128.2051ms/op), total: 299 (1 thr)
Rate: 83.00/thr ( 8.30/sec =120.4819ms/op), total: 382 (1 thr)
Rate: 93.00/thr ( 9.30/sec =107.5269ms/op), total: 475 (1 thr)
==============================================================================================================
2. Update changelog encryption along with the Certificate renewal
2-1. Export changelog db on Master servers.
$ ldapmodify [...]
dn: cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config
changetype: modify
add: nsds5Task
nsds5Task: CL2LDIF
$ ldapmodify [...]
dn: cn=replica,cn=dc\3Dtest\2Cdc\3Dcom,cn=mapping tree,cn=config
changetype: modify
add: nsds5Task
nsds5Task: CL2LDIF
Monitor the error log /var/log/dirsrv/slapd-master[12]/errors to check the export is successfully finished.
[...] NSMMReplicationPlugin - Beginning changelog export of replica "5af4cd84-1dd211b2-b4b8f8dd-b6310000"
[...] NSMMReplicationPlugin - Finished changelog export of replica "5af4cd84-1dd211b2-b4b8f8dd-b6310000"
===========================================================================================================
[root@snmaptest /]# ldapmodify -D "cn=directory manager" -w Secret123 -p 30100 -x -h localhost << EOF
dn: cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config
changetype: modify
add: nsds5Task
nsds5Task: CL2LDIF
EOF
[root@snmaptest changelog]# tail -f /var/log/dirsrv/slapd-M1/errors
[22/Sep/2011:14:49:50 +051800] - All database threads now stopped
[22/Sep/2011:14:49:50 +051800] - slapd stopped.
[22/Sep/2011:14:51:35 +051800] - 389-Directory/1.2.9.11 B2011.259.2023 starting up
[22/Sep/2011:14:51:35 +051800] attrcrypt - _back_crypt_cipher_init: No symmetric key found for cipher AES, attempting to create one...
[22/Sep/2011:14:51:36 +051800] - slapd started. Listening on All Interfaces port 30100 for LDAP requests
[22/Sep/2011:14:51:36 +051800] - Listening on All Interfaces port 30101 for LDAPS requests
[22/Sep/2011:15:01:25 +051800] NSMMReplicationPlugin - Beginning total update of replica "agmt="cn=M1_to_M2" (snmaptest:30103)".
[22/Sep/2011:15:01:28 +051800] NSMMReplicationPlugin - Finished total update of replica "agmt="cn=M1_to_M2" (snmaptest:30103)". Sent 160 entries.
[22/Sep/2011:15:07:45 +051800] NSMMReplicationPlugin - Beginning total update of replica "agmt="cn=M1_to_M4" (snmaptest:30107)".
[22/Sep/2011:15:07:48 +051800] NSMMReplicationPlugin - Finished total update of replica "agmt="cn=M1_to_M4" (snmaptest:30107)". Sent 160 entries.
[22/Sep/2011:15:30:57 +051800] NSMMReplicationPlugin - Beginning changelog export of replica "6e06a173-e4fa11e0-9528d0a9-2fb8bf08"
[22/Sep/2011:15:30:58 +051800] NSMMReplicationPlugin - Finished changelog export of replica "6e06a173-e4fa11e0-9528d0a9-2fb8bf08"
===========================================================================================================
2-2. Check the exported changelog file on Master servers (one ldif file per changelog db).
# ls /var/lib/dirsrv/slapd-master[12]/changelogdb/*.ldif
/var/lib/dirsrv/slapd-master[12]/changelogdb/[...].ldif
Changes in each changelog is base64 encoded. E.g.,
changetype: add
replgen: 4d2b599c000000010000
csn: 4d2b5a9c000000020000
nsuniqueid: c63f9f01-1dd111b2-ad4a8494-47bd0000
parentuniqueid: 40bbef65-1dd211b2-b4baf8dd-b6310000
dn: uid=j0user0,ou=People,dc=test,dc=com
change:: YWRkOiB1aWQKdWlkOiBqMHVzZXIwCi0KYWRkOiBnaXZlbk5hbWUKZ2l2ZW5OYW1lOiBq
aWppMAotCmFkZDogb2JqZWN0Q2xhc3MKb2JqZWN0Q2xhc3M6IHRvcApvYmplY3RDbGFzczogcGVy
c29uCm9iamVjdENsYXNzOiBvcmdhbml6YXRpb25hbFBlcnNvbgpvYmplY3RDbGFzczogaW5ldG9y
Z3BlcnNvbgotCmFkZDogc24Kc246IHVzZXIwCi0KYWRkOiBjbgpjbjogamlqaTAgdXNlcjAKLQph
ZGQ6IHVzZXJQYXNzd29yZAp1c2VyUGFzc3dvcmQ6IHtTU0hBfUI5K25pWTkzdE9XYzlhTHF5djdv
b1MwVUJrdktlaUgzZndxNGFBPT0KLQphZGQ6IGNyZWF0b3JzTmFtZQpjcmVhdG9yc05hbWU6IGNu
PWRpcmVjdG9yeSBtYW5hZ2VyCi0KYWRkOiBtb2RpZmllcnNOYW1lCm1vZGlmaWVyc05hbWU6IGNu
PWRpcmVjdG9yeSBtYW5hZ2VyCi0KYWRkOiBjcmVhdGVUaW1lc3RhbXAKY3JlYXRlVGltZXN0YW1w
OiAyMDExMDExMDE5MTQzNloKLQphZGQ6IG1vZGlmeVRpbWVzdGFtcAptb2RpZnlUaW1lc3RhbXA6
IDIwMTEwMTEwMTkxNDM2WgotCmFkZDogbnNVbmlxdWVJZApuc1VuaXF1ZUlkOiBjNjNmOWYwMS0x
ZGQxMTFiMi1hZDRhODQ5NC00N2JkMDAwMAotCmFkZDogdW5oYXNoZWQjdXNlciNwYXNzd29yZAp1
bmhhc2hlZCN1c2VyI3Bhc3N3b3JkOiBqMHVzZXIwCi0K
Decode the change and make sure it is not encrypted:
add: uid
uid: juser0
-
add: givenName
givenName: jiji
-
add: objectClass
[...]
-
add: nsUniqueId
nsUniqueId: af00b181-1dd111b2-b4bbf8dd-b6310000
-
add: unhashed#user#password
unhashed#user#password: juser0
===========================================================
[root@snmaptest changelog]# ls /var/lib/dirsrv/slapd-M1/db/changelog/
6e06a173-e4fa11e0-9528d0a9-2fb8bf08_4e7afb14000000010000.db4 6e06a173-e4fa11e0-9528d0a9-2fb8bf08.ldif 6e06a173-e4fa11e0-9528d0a9-2fb8bf08.sema DBVERSION
ldif file contents
changetype: add
replgen: 4e7afb14000000010000
csn: 4e7b0243000000010000
nsuniqueid: a4461f82-e4fe11e0-9528d0a9-2fb8bf08
parentuniqueid: 6cd57425-e4fa11e0-9528d0a9-2fb8bf08
dn: uid=amsharma2,ou=people,dc=example,dc=com
change:: YWRkOiBjbgpjbjogYW1zCi0KYWRkOiBzbgpzbjogYW1zCi0KYWRkOiBnaXZlbk5hbWUKZ
2l2ZW5OYW1lOiBhbXMKLQphZGQ6IG9iamVjdENsYXNzCm9iamVjdENsYXNzOiB0b3AKb2JqZWN0Q2
xhc3M6IHBlcnNvbgpvYmplY3RDbGFzczogb3JnYW5pemF0aW9uYWxQZXJzb24Kb2JqZWN0Q2xhc3M
6IGluZXRPcmdQZXJzb24KLQphZGQ6IHVpZAp1aWQ6IGFtcwp1aWQ6IGFtc2hhcm1hMgotCmFkZDog
bWFpbAptYWlsOiBhbXNAZXhhbXBsZS5jb20KLQphZGQ6IHVzZXJQYXNzd29yZAp1c2VyUGFzc3dvc
mQ6OiBlMU5UU0VGOVF6bE9hMVpDZDNWa2EzRXJhVU0zT0VkTVVGVk5OM2MyTWtRMVdsbHVMMFZJYU
ZocVMxRTlQUT0KID0KLQphZGQ6IGNyZWF0b3JzTmFtZQpjcmVhdG9yc05hbWU6IGNuPWRpcmVjdG9
yeSBtYW5hZ2VyCi0KYWRkOiBtb2RpZmllcnNOYW1lCm1vZGlmaWVyc05hbWU6IGNuPWRpcmVjdG9y
eSBtYW5hZ2VyCi0KYWRkOiBjcmVhdGVUaW1lc3RhbXAKY3JlYXRlVGltZXN0YW1wOiAyMDExMDkyM
jA5MzkxNVoKLQphZGQ6IG1vZGlmeVRpbWVzdGFtcAptb2RpZnlUaW1lc3RhbXA6IDIwMTEwOTIyMD
kzOTE1WgotCmFkZDogbnNVbmlxdWVJZApuc1VuaXF1ZUlkOiBhNDQ2MWY4Mi1lNGZlMTFlMC05NTI
4ZDBhOS0yZmI4YmYwOAotCmFkZDogdW5oYXNoZWQjdXNlciNwYXNzd29yZAp1bmhhc2hlZCN1c2Vy
I3Bhc3N3b3JkOiBhbXNhbXNhbXMKLQo=
Decoded contents
[root@snmaptest changelog]# base64 -d test.b64
add: cn
cn: ams
-
add: sn
sn: ams
-
add: givenName
===========================================================
2-3. Recommend to back up DBs on each server
# Stop the servers
# /usr/lib[64]/dirsrv/slapd-ID/db2bak
2-4. Stop the server and disable changelog encryption.
Remove these config entries from dse.ldif (2 entries per backend -- suffix):
dn: cn=3DES,cn=encrypted attribute keys,cn=<backend>,cn=ldbm database,cn=plugins,cn=config
dn: cn=AES,cn=encrypted attribute keys,cn=<backend>,cn=ldbm database,cn=plugins,cn=config
Remove these config attr values from cn=changelog5,cn=config
nsslapd-encryptionalgorithm: AES
nsSymmetricKey:: LrKrvjtihBJA8G5aBohkABd2pUyM7iwn2EO1Y7QpU7iJhHDsfV+j12prQBp3
[...]
===========================================================
[root@snmaptest changelog]# service dirsrv stop
Shutting down dirsrv:
M1... [ OK ]
M2... [ OK ]
M3... [ OK ]
M4... [ OK ]
snmaptest... [ OK ]
[root@snmaptest changelog]#
Removed specified enteries.
=============================================================
2-5. Renew the server certificate
2-6. Stop Master servers and set nsslapd-encryptionalgorithm. The allowed value is AES or 3DES.
dn: cn=changelog5,cn=config
[...]
nsslapd-encryptionalgorithm: AES
2-7. Restart the servers and import the changelog
$ ldapmodify [...]
dn: cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config
changetype: modify
add: nsds5Task
nsds5Task: LDIF2CL
$ ldapmodify [...]
dn: cn=replica,cn=dc\3Dtest\2Cdc\3Dcom,cn=mapping tree,cn=config
changetype: modify
add: nsds5Task
nsds5Task: LDIF2CL
======================================================================================
[root@snmaptest fourwaymmr]# service dirsrv stop
Shutting down dirsrv:
M1... [ OK ]
M2... [ OK ]
M3... [ OK ]
M4... [ OK ]
snmaptest... [ OK ]
[root@snmaptest fourwaymmr]# vim /etc/dirsrv/slapd-M1/dse.ldif
[root@snmaptest fourwaymmr]# service dirsrv start
Starting dirsrv:
M1... [ OK ]
M2... [ OK ]
M3... [ OK ]
M4... [ OK ]
snmaptest... [ OK ]
[root@snmaptest fourwaymmr]# ldapmodify -D "cn=directory manager" -w Secret123 -p 30100 -x -h localhost << EOF
> dn: cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config
> changetype: modify
> add: nsds5Task
> nsds5Task: LDIF2CL
> EOF
modifying entry "cn=replica,cn=dc\3Dexample\2Cdc\3Dcom,cn=mapping tree,cn=config"
[root@snmaptest fourwaymmr]# tail -f /var/log/dirsrv/slapd-M1
tail: error reading `/var/log/dirsrv/slapd-M1': Is a directory
tail: /var/log/dirsrv/slapd-M1: cannot follow end of this type of file; giving up on this name
[root@snmaptest fourwaymmr]# tail -f /var/log/dirsrv/slapd-M1/errors
[22/Sep/2011:16:57:23 +051800] - slapd shutting down - closing down internal subsystems and plugins
[22/Sep/2011:16:57:24 +051800] - Waiting for 4 database threads to stop
[22/Sep/2011:16:57:24 +051800] - All database threads now stopped
[22/Sep/2011:16:57:24 +051800] - slapd stopped.
[22/Sep/2011:16:58:06 +051800] - 389-Directory/1.2.9.11 B2011.259.2023 starting up
[22/Sep/2011:16:58:06 +051800] attrcrypt - _back_crypt_cipher_init: No symmetric key found for cipher AES, attempting to create one...
[22/Sep/2011:16:58:06 +051800] - slapd started. Listening on All Interfaces port 30100 for LDAP requests
[22/Sep/2011:16:58:06 +051800] - Listening on All Interfaces port 30101 for LDAPS requests
[22/Sep/2011:16:59:15 +051800] NSMMReplicationPlugin - Beginning changelog import of replica "6e06a173-e4fa11e0-9528d0a9-2fb8bf08"
[22/Sep/2011:16:59:26 +051800] NSMMReplicationPlugin - Finished changelog import of replica "6e06a173-e4fa11e0-9528d0a9-2fb8bf08"
======================================================================================
Monitor the error log /var/log/dirsrv/slapd-master[12]/errors to check the import is successfully finished.
[...] NSMMReplicationPlugin - Beginning changelog import of replica "5af4cd82-1dd211b2-b4b8f8dd-b6310000"
[...] NSMMReplicationPlugin - Finished changelog import of replica "5af4cd82-1dd211b2-b4b8f8dd-b6310000"
2-8. For testing, modify something on the both masters and check the change is replicated to the replicas.
Hence Verified.
|
Description of problem: Email discussion: =========================== Basic Cert renewal scenario =========================== Assume the server already sets up the both attrcrypt and encchangelog. 0. Before the cert is renewed, customers is supposed to do export the DB contents (db2ldif -E ...). # ./db2ldif -n backend_instance -E Exported ldif file: /var/lib/dirsrv/slapd-server_ID/ldif/server_ID-backend_instance-date.ldif 1. Stop the server service dirsrv stop server_ID 2. Renew the server cert. 3. Edit /etc/dirsrv/slapd-ID/dse.ldif 3-1. Remove attrcrypt entries: dn: cn=3DES,cn=encrypted attribute keys,cn=backend_instance,cn=ldbm database,cn=plugins,cn=config objectClass: top objectClass: extensibleObject cn: 3DES nsSymmetricKey:: EGXb3F59jqZzOs6gU/MmuP1FS6uesD9i1Y3riJjLsYrEDVloTd98lmOcKsg1 u0uR8qT4uEnfaHnZY1Y7F4tcaaP9sQiiM2V9BfDBT9e19AqRHAO5vt2LrujT5Js4ZoO0lPXl8o3v VaH4Ju10wWs6jSoTyKSzGSHFWd/nrZq0wLg= dn: cn=AES,cn=encrypted attribute keys,cn=backend_instance,cn=ldbm database,cn=plugins ,cn=config objectClass: top objectClass: extensibleObject cn: AES nsSymmetricKey:: CdPY1U7A6rrt1mgPy60D+0MZ4Am8EAKEp/DL9uXJrdvpBO0+oiH6IxNDtieR nFjyG2uKEdy2gwcUGLU3TXWOoqwfANIP1HPWmpWc1nQ6aqMh2OJqvnZhL3j+kkDCDV6y68YyYrK2 16VJiWEsWnENXqnqpp8bnUQhmo9O1PlU8eI= 3-2. Remove nsSymmetricKey from cn=changelog5,cn=config dn: cn=changelog5,cn=config [...] nsSymmetricKey:: JP4WSyHmkdae7pzfhoCppdk2+09o+i2KTV3CYqjb00z9igQwNf+24v5jh2I7 UFod/YJtMpuWN/awTkrXGsCWPR3lZEHTWlNG+peXcFVAV2hWTrcPPjXMlsL++wS7urzpW/BFD10v kK7WEtzD50iHS0BMxYvWVzyblDk6jybOLD4= 4. Start the server to generate a new symmetric key. service dirsrv start server_ID 5. Run online import ./ldif2db.pl -D 'cn=directory manager' -w password -n backend_instance -E -i /var/lib/dirsrv/slapd-server_ID/ldif/server_ID-backend_instance-date.ldif or stop the server then run standalone import. service dirsrv stop server_ID ./ldif2db -n backend_instance -E -i /var/lib/dirsrv/slapd-server_ID/ldif/server_ID-backend_instance-date.ldif 6. Regenerate the changelog. I did once removing a replica and recreated it. Could there be any easier/better way for regenerating a changelog? ===================== Rich Megginson wrote: ===================== Sounds like we need a way to either dump/import a changelog or "re-encrypt" it in place.