1171308 – replication not working for "add: userPassword" LDAP operation

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1171308 - replication not working for "add: userPassword" LDAP operation

Summary: replication not working for "add: userPassword" LDAP operation

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 6
Classification:	Red Hat
Component:	389-ds-base
Sub Component:
Version:	6.3
Hardware:	Unspecified
OS:	Unspecified
Priority:	high
Severity:	high
Target Milestone:	rc
Target Release:	---
Assignee:	Noriko Hosoi
QA Contact:	Viktor Ashirov
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2014-12-05 21:15 UTC by Thang Nguyen
Modified:	2020-09-13 21:01 UTC (History)
CC List:	8 users (show)
Fixed In Version:	389-ds-base-1.2.11.15-51.el6
Doc Type:	Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:	2015-07-22 06:36:27 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)
replication log (9.30 KB, text/plain) 2014-12-05 21:15 UTC, Thang Nguyen	no flags	Details
consumer access log (124.91 KB, text/plain) 2014-12-05 23:37 UTC, Thang Nguyen	no flags	Details
View All

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Github	389ds 389-ds-base issues 1084	0	None	None	None	2020-09-13 21:01:10 UTC
Red Hat Product Errata	RHBA-2015:1326	0	normal	SHIPPED_LIVE	389-ds-base bug fix and enhancement update	2015-07-20 17:53:07 UTC

Description Thang Nguyen 2014-12-05 21:15:11 UTC

Created attachment 965227 [details]
replication log

Description of problem:
In a replication (master-master or master-consumer) environment, when doing a 

dn: cn=user1,dc=ou,dc=o
changetype: modify
add: userPassword
userPassword: xxxxxxx

on the master server, the userPassword attribute/value is added to the master but cannot be replicated to the consumer.

I get the following error in the error log:

[04/Dec/2014:10:12:58 -1000] NSMMReplicationPlugin - agmt="cn=ldp93-to-ldp92-636" (ldp92:636): Consumer failed to replay change (uniqueid 5b2cfd01-7bf011e4-85c0de15-2b200dc9, CSN 5480c04f0000005d0000): Protocol error (2). Will retry later.

Attached is the replication log (nsslapd-errorlog-level: 8192) for this operation.

This happens on version 1.2.11.15-48.el6_6 but NOT on 1.2.11.15-34.el6_5.

Version-Release number of selected component (if applicable):
389-ds-base.x86_64 1.2.11.15-48.el6_6


Steps to Reproduce:
1. configure a master-consumer replication environment
2. create any new ldap record without userPassword attribute
3. do "add: userPassword" as mentioned in the description of problem section.
4. check the error log

Comment 2 Noriko Hosoi 2014-12-05 22:40:58 UTC

The error is reported when the master gets CONN_NOT_CONNECTED:
  /* We lost the connection - enter backoff state */

Could you provide the error log as well as the access log of the consumer server?

Also, if you run ldapsearch against the master and the consumer, what's returned? (please see also https://access.redhat.com/documentation/en-US/Red_Hat_Directory_Server/9.0/html/Administration_Guide/ldap-clients-ssl.html for running ldapsearch over SSL.)

ldapsearch -x -H "ldaps://YOUR_HOST:636" -D YOUR_DIR_MGR -w YOUR_PW -b "cn=user1,dc=ou,dc=o" userPassword

Comment 3 Thang Nguyen 2014-12-05 23:37:44 UTC

Created attachment 965276 [details]
consumer access log

Comment 4 Thang Nguyen 2014-12-05 23:41:47 UTC

I attached the consumer access log.  There's nothing in the consumer error log around this time.  When I do an ldapsearch, I see the userPassword attribute on the master but not on the consumer server.

Comment 5 Noriko Hosoi 2014-12-06 01:12:39 UTC

Thanks for the input.

It looks the mod sent to the consumer was empty. 

[04/Dec/2014:10:12:58 -1000] conn=2662 op=7 EXT oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop"
[04/Dec/2014:10:12:58 -1000] conn=2662 op=7 RESULT err=0 tag=120 nentries=0 etime=0
[04/Dec/2014:10:12:58 -1000] conn=2662 op=8 MOD dn="uhEntry=d78930247890123479012347801237489123,ou=People,dc=hawaii,dc=edu" authzid="(null)", no values given
[04/Dec/2014:10:12:58 -1000] conn=2662 op=8 RESULT err=2 tag=103 nentries=0 etime=0
[04/Dec/2014:10:12:58 -1000] conn=2662 op=9 MOD dn="uhEntry=d78930247890123479012347801237489123,ou=People,dc=hawaii,dc=edu"
[04/Dec/2014:10:12:58 -1000] conn=2662 op=9 RESULT err=0 tag=103 nentries=0 etime=0 csn=5480c04f0001005d0000

If you run "dbscan -f /path/to/yourMastersChangelogDir/yourChangeLog.db4" against your master server, do you see the contents of the modify operation?

Another question, is the symptom specific to userPassword?  And does it occur every time you add userPassword with modify?

Thanks.

Comment 6 Thang Nguyen 2014-12-06 01:36:39 UTC

I see the contents in the dbscan output.

To clarify the issue:

Replication does NOT work with the following LDIF:

dn: cn=user1,dc=ou,dc=o
changetype: modify
add: userPassword
userPassword: {SSHA}xxxxxxx

However, I just tested and replication does work with this LDIF:

dn: cn=user1,dc=ou,dc=o
changetype: modify
add: userPassword
userPassword: xxxxxxx

Seems like the "{SSHA}" causes the problem.

Comment 7 Thang Nguyen 2014-12-06 01:44:36 UTC

I see the contents in the dbscan output.

To clarify the issue:

Replication does NOT work with the following LDIF:

dn: cn=user1,dc=ou,dc=o
changetype: modify
add: userPassword
userPassword: {SSHA}xxxxxxx

However, I just tested and replication does work with this LDIF:

dn: cn=user1,dc=ou,dc=o
changetype: modify
add: userPassword
userPassword: xxxxxxx

Seems like the "{SSHA}" causes the problem.

Comment 8 Thang Nguyen 2014-12-06 01:46:42 UTC

I want to add that replication works with "replace" that has {SSHA}:

dn: cn=user1,dc=ou,dc=o
changetype: modify
replace: userPassword
userPassword: {SSHA}xxxxxxx

Comment 9 Noriko Hosoi 2014-12-06 02:00:44 UTC

Thanks, Thang.

I could reproduce the problem.

Could you please confirm one thing?

This fails with Protocol error (2).
dn: cn=user1,dc=ou,dc=o
changetype: modify
add: userPassword
userPassword: {SSHA}xxxxxxx

This modify-add is successfully replicated to the consumer.
dn: cn=user1,dc=ou,dc=o
changetype: modify
add: userPassword
userPassword: xxxxxxx

This modify-replace is successfully replicated, too.  Right?
dn: cn=user1,dc=ou,dc=o
changetype: modify
replace: userPassword
userPassword: {SSHA}xxxxxxx

Comment 10 Thang Nguyen 2014-12-06 02:02:20 UTC

That's correct.  Thanks Noriko!

Comment 11 Sankar Ramalingam 2014-12-08 09:02:07 UTC

I tested this on RHEL6.6 and its reproducible as mentioned above in comment #9.
Build tested - 389-ds-base-1.2.11.15-48

LDAP Modify-add with hashed password FAILS:

ldapmodify -x -p 1189 -h localhost -D "cn=Directory Manager" -w Secret123
dn: cn=ttusr2,ou=people,dc=passsync,dc=com
changetype: modify
add: userPassword
userPassword: {SSHA}U2VjcmV0MTIzNA==

ldapsearch -LLLx -p 1189 -h localhost -D "cn=Directory Manager" -w Secret123 -b "dc=passsync,dc=com" cn=ttusr2 userPassword
dn: cn=ttusr2,ou=People,dc=passsync,dc=com
userPassword:: e1NTSEF9VTJWamNtVjBNVEl6TkE9PQ==

ldapsearch -LLLx -p 1289 -h localhost -D "cn=Directory Manager" -w Secret123 -b "dc=passsync,dc=com" cn=ttusr2 userPassword
dn: cn=ttusr2,ou=People,dc=passsync,dc=com

LDAP Modify-replace: it works fine.

[root@iceman ~]# ldapmodify -x -p 1189 -h localhost -D "cn=Directory Manager" -w Secret123 << EOF
dn: cn=ttusr3,ou=people,dc=passsync,dc=com
changetype: modify
replace: userPassword
userPassword: {SSHA}U2VjcmV0MTIzNA==
EOF

modifying entry "cn=ttusr3,ou=people,dc=passsync,dc=com"

[root@iceman ~]# ldapsearch -LLLx -p 1289 -h localhost -D "cn=Directory Manager" -w Secret123 -b "dc=passsync,dc=com" cn=ttusr3 userPassword
dn: cn=ttusr3,ou=People,dc=passsync,dc=com
userPassword:: e1NTSEF9VTJWamNtVjBNVEl6TkE9PQ==

[root@iceman ~]# ldapsearch -LLLx -p 1189 -h localhost -D "cn=Directory Manager" -w Secret123 -b "dc=passsync,dc=com" cn=ttusr3 userPassword
dn: cn=ttusr3,ou=People,dc=passsync,dc=com
userPassword:: e1NTSEF9VTJWamNtVjBNVEl6TkE9PQ==


With modify-add with regular password: works fine
[root@iceman ~]# ldapmodify -x -p 1189 -h localhost -D "cn=Directory Manager" -w Secret123 << EOF
dn: cn=ttusr4,ou=people,dc=passsync,dc=com
changetype: modify
add: userPassword
userPassword: Secret1234            
EOF

modifying entry "cn=ttusr4,ou=people,dc=passsync,dc=com"

[root@iceman ~]# ldapsearch -LLx -p 1189 -h localhost -D "cn=Directory Manager" -w Secret123 -b "dc=passsync,dc=com" cn=ttusr4 userPassword
dn: cn=ttusr4,ou=People,dc=passsync,dc=com
userPassword:: e1NTSEF9UU95cXlWMnRFTDk3Ukp3ekNFclFIa2o5SllzTjlyajhnazczYWc9PQ=
 =

[root@iceman ~]# ldapsearch -LLLx -p 1289 -h localhost -D "cn=Directory Manager" -w Secret123 -b "dc=passsync,dc=com" cn=ttusr4 userPassword
dn: cn=ttusr4,ou=People,dc=passsync,dc=com
userPassword:: e1NTSEF9UU95cXlWMnRFTDk3Ukp3ekNFclFIa2o5SllzTjlyajhnazczYWc9PQ=
 =

Comment 12 Noriko Hosoi 2014-12-10 22:43:58 UTC

Fixed upstream.

Comment 13 Victoriano Giralt 2015-02-17 19:06:06 UTC

We have also been bitten by this bug. I may not post the changelog because it contains PII (Personal Identifiable Information).

We have found that any other attributes that go in the same change record with the added base64 encoded hashed userPassword do not replicate as well.

Is there any time frame for new RPMs with the fix included?

We have had to implement a very ugly hack to our user provisioning system. And we have to provision quite a number almost every day.

Thanks.

Comment 17 Sankar Ramalingam 2015-03-27 03:35:49 UTC

Thanks Noriko for the clarification. Marking the bug as Verified.

Comment 18 Chris Samuel 2015-05-28 05:01:37 UTC

Confirming this here at VLSCI with:

[root@ldap2 slapd-ldap2]# yum info 389-ds-base
Loaded plugins: etckeeper, product-id, rhnplugin, security, subscription-manager
This system is receiving updates from RHN Classic or RHN Satellite.
Installed Packages
Name        : 389-ds-base
Arch        : x86_64
Version     : 1.2.11.15
Release     : 50.el6_6
Size        : 4.9 M
Repo        : installed
From repo   : rhel-x86_64-server-6
Summary     : 389 Directory Server (base)
URL         : http://port389.org/
License     : GPLv2 with exceptions
Description : 389 Directory Server is an LDAPv3 compliant server.  The base package includes
            : the LDAP server and command line utilities for server administration.

Comment 19 errata-xmlrpc 2015-07-22 06:36:27 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-1326.html

Note You need to log in before you can comment on or make changes to this bug.