Bug 552421 - Cannot log into admin server after upgrade (fedora-ds-admin-1.1.6 -> 389-admin-1.1.9
Summary: Cannot log into admin server after upgrade (fedora-ds-admin-1.1.6 -> 389-admi...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: 389
Classification: Retired
Component: Admin
Version: 1.2.1
Hardware: All
OS: Linux
high
high
Target Milestone: ---
Assignee: Rich Megginson
QA Contact: Viktor Ashirov
URL:
Whiteboard:
Depends On:
Blocks: 434915 389_1.2.6
TreeView+ depends on / blocked
 
Reported: 2010-01-05 00:11 UTC by Orion Poplawski
Modified: 2015-12-07 16:39 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-12-07 16:39:40 UTC


Attachments (Terms of Use)
patch (4.56 KB, patch)
2010-01-11 20:14 UTC, Rich Megginson
nkinder: review+
Details | Diff

Description Orion Poplawski 2010-01-05 00:11:55 UTC
Description of problem:

Testing update of our directory server from fedora-ds-admin-1.1.6 -> 389-admin-1.1.9.  After update, cannot log into admin server.  setup-ds-admin.pl complains about the password as well:

Could not authenticate as user 'uid=admin, ou=Administrators, ou=TopologyManagement, o=NetscapeRoot' to server 'ldap://ldap.cora.nwra.com:389/o=NetscapeRoot'.  Error: Invalid credentials

Please try again, in case you mis-typed something.

Version-Release number of selected component (if applicable):
389-admin-1.1.9-1.el5

slapd access:
[04/Jan/2010:17:09:53 -0700] conn=32 fd=65 slot=65 connection from 192.168.0.154 to 192.168.0.154
[04/Jan/2010:17:09:53 -0700] conn=32 op=0 BIND dn="" method=128 version=3
[04/Jan/2010:17:09:53 -0700] conn=32 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn=""
[04/Jan/2010:17:09:53 -0700] conn=32 op=1 BIND dn="uid=admin, ou=Administrators, ou=TopologyManagement, o=NetscapeRoot" method=128 version=3
[04/Jan/2010:17:09:53 -0700] conn=32 op=2 UNBIND
[04/Jan/2010:17:09:53 -0700] conn=32 op=2 fd=65 closed - U1
[04/Jan/2010:17:09:53 -0700] conn=32 op=1 RESULT err=49 tag=97 nentries=0 etime=0

Comment 1 Rich Megginson 2010-01-05 01:54:47 UTC
Does ldapsearch work from the command line? e.g.

ldapsearch -x -D "uid=admin, ou=Administrators, ou=TopologyManagement, o=NetscapeRoot" -w "yourpassword" -s base -b ""

Comment 2 Orion Poplawski 2010-01-05 15:44:36 UTC
Nope:

ldap_bind: Invalid credentials (49)

But it does work with admin 1.1.6 and ds 1.1.2 which is what I'm running in production.

I'm testing by copying over /etc/dirsrv and /var/lib/dirsrv from my production machine to a test machine and upgrading the software there.  Is it possible I'm not copying a needed file over?  System seems to work fine before the upgrade though.

Note that I'm also updating ds-base from 1.1.2 -> 1.2.4 as part of this upgrade.

dn:
objectClass: top
namingContexts: dc=nwra, dc=com
namingContexts: o=netscaperoot
supportedExtension: 2.16.840.1.113730.3.5.7
supportedExtension: 2.16.840.1.113730.3.5.8
supportedExtension: 2.16.840.1.113730.3.5.3
supportedExtension: 2.16.840.1.113730.3.5.5
supportedExtension: 2.16.840.1.113730.3.5.6
supportedExtension: 2.16.840.1.113730.3.5.9
supportedExtension: 2.16.840.1.113730.3.5.4
supportedExtension: 1.3.6.1.4.1.1466.20037
supportedExtension: 1.3.6.1.4.1.4203.1.11.1
supportedControl: 2.16.840.1.113730.3.4.2
supportedControl: 2.16.840.1.113730.3.4.3
supportedControl: 2.16.840.1.113730.3.4.4
supportedControl: 2.16.840.1.113730.3.4.5
supportedControl: 1.2.840.113556.1.4.473
supportedControl: 2.16.840.1.113730.3.4.9
supportedControl: 2.16.840.1.113730.3.4.16
supportedControl: 2.16.840.1.113730.3.4.15
supportedControl: 2.16.840.1.113730.3.4.17
supportedControl: 2.16.840.1.113730.3.4.19
supportedControl: 1.3.6.1.4.1.42.2.27.8.5.1
supportedControl: 1.3.6.1.4.1.42.2.27.9.5.2
supportedControl: 2.16.840.1.113730.3.4.14
supportedControl: 2.16.840.1.113730.3.4.20
supportedControl: 1.3.6.1.4.1.1466.29539.12
supportedControl: 2.16.840.1.113730.3.4.12
supportedControl: 2.16.840.1.113730.3.4.18
supportedControl: 2.16.840.1.113730.3.4.13
supportedSASLMechanisms: EXTERNAL
supportedSASLMechanisms: LOGIN
supportedSASLMechanisms: DIGEST-MD5
supportedSASLMechanisms: PLAIN
supportedSASLMechanisms: ANONYMOUS
supportedSASLMechanisms: CRAM-MD5
supportedSASLMechanisms: GSSAPI
supportedLDAPVersion: 2
supportedLDAPVersion: 3
vendorName: Fedora Project
vendorVersion: Fedora-Directory/1.1.2 B2008.259.1552
dataversion: 020091223000051020091223000051
netscapemdsuffix: cn=ldap://dc=ldap,dc=cora,dc=nwra,dc=com:389

Comment 3 Orion Poplawski 2010-01-05 15:50:07 UTC
I can still bind to the serve with "cn=Directory Manager".

Comment 4 Orion Poplawski 2010-01-05 15:56:23 UTC
I still have the entry, and it is exactly the same as on the production server:

# admin, Administrators, TopologyManagement, NetscapeRoot
dn: uid=admin, ou=Administrators, ou=TopologyManagement, o=NetscapeRoot
givenName: Configuration
objectClass: top
objectClass: person
objectClass: organizationalperson
objectClass: inetorgperson
uid: admin
cn: Configuration Administrator
sn: Administrator
userPassword:: e1XXXXXXXXXXXXXXXXXXXXX==

(Omitted the password)

Comment 5 Rich Megginson 2010-01-05 17:02:32 UTC
Can you verify what version of 389-ds-base you are using?
This sounds suspiciously like https://bugzilla.redhat.com/show_bug.cgi?id=518520 which was fixed in 389-ds-base 1.2.2

What encoding does the password use?  The userPassword is base64 encoded - you can see the actual value by using python:
python
> import base64
> base64.b64decode('the value')

Is the password using {SHA} or {SSHA} or something else?

Comment 6 Orion Poplawski 2010-01-05 17:34:14 UTC
It's using {SHA}.

389-ds-base-1.2.4-1.el5

Comment 7 Rich Megginson 2010-01-05 17:48:00 UTC
ok - try this:

python
> import base64
> import hashlib
> sha = hashlib.sha1('your password')
> base64.b64encode(sha.digest())

Does the value it prints out match your password (that is, the stuff that comes after the {SHA} in your password)?

Comment 8 Orion Poplawski 2010-01-05 18:04:13 UTC
Running through the debugger, it's getting called with:

Breakpoint 1, sha_pw_cmp (userpwd=0xa118918 "XXXXXXXX", 
    dbpwd=0xa181f95 "XXXXXXXXXXXXXXXXX", shaLen=20)
    at ldap/servers/plugins/pwdstorage/sha_pwd.c:72             

Resulting in the error:

$10 = 0x16ac80 "pw_cmp: %s userPassword \"%s\" is the wrong length or is not properly encoded BASE64\n"

Which isn't getting logged because no debugging is turned on.  How can I turn on debugging so I can see these messages?

Looks like shaLen is hardcoded to 20?  I assume it should match the length of dbpwd?

I've obfuscated the passwords, but dbpwd is 29 characters, and the last character is a newline, which is strange, but perhaps valid in the hash.

(gdb) print dbpwd[28]
$14 = 10 '\n'
(gdb) print dbpwd[29]
$15 = 0 '\0'

Comment 9 Orion Poplawski 2010-01-05 18:09:21 UTC
(In reply to comment #7)
> Does the value it prints out match your password (that is, the stuff that comes
> after the {SHA} in your password)?  


It does, minus the newline character.

Comment 10 Rich Megginson 2010-01-05 19:56:26 UTC
> How can I turn on debugging so I can see these messages?

These messages are logged at the Plug-in debugging log level - see http://directory.fedoraproject.org/wiki/FAQ#Troubleshooting

> Looks like shaLen is hardcoded to 20?

Right.  The SHA algorithm always produces 20 bytes.

> I assume it should match the length of dbpwd?

Yes and no.  dbpwd is the base64 encoded sha hash of the password.  If you base64 encode 20 bytes, you will get 28 bytes.

In python:
>>> len(sha.digest())
20
>>> len(base64.b64encode(sha.digest()))
28

So dbpwd should be 28 bytes.  I think the problem is that the password should not contain the trailing newline.  Do you remember what version of fedora-ds-base and fedora-ds-admin were used to create the initial password?

Comment 11 Orion Poplawski 2010-01-05 20:39:23 UTC
(In reply to comment #10)
> So dbpwd should be 28 bytes.  I think the problem is that the password should
> not contain the trailing newline.  Do you remember what version of
> fedora-ds-base and fedora-ds-admin were used to create the initial password?  

Quite likely fedora-ds-base-1.1.0 and fedora-ds-admin-1.1.2.  I'm currently running base-1.1.2 and admin-1.1.6.

I removed the trailing 'Cg==' from the userPassword which corresponded to the newline character and now I can authenticate as the admin user.  I made the change on the production server and can still authenticate there as well.

I don't know if it's worth it or possible to try to fix this automatically during upgrades, of it I'm the only one who ended up with a newline encoded in the password somehow.

Thanks for all the help.

Comment 12 Rich Megginson 2010-01-05 20:53:04 UTC
Thanks for confirming my suspicions about the newline.  I think this was a bug that was fixed quite some time ago, that it didn't trim the trailing newline when creating the password.  I'd rather not attempt to fix the userPassword value.  I think the solution that will cause the least disruption will be to change the sha_pw_cmp routine to ignore a trailing newline.

Comment 13 Rich Megginson 2010-01-08 21:36:59 UTC
I'm a bit confused here - you say https://bugzilla.redhat.com/show_bug.cgi?id=552421#c8

I've obfuscated the passwords, but dbpwd is 29 characters, and the last
character is a newline, which is strange, but perhaps valid in the hash.

(gdb) print dbpwd[28]
$14 = 10 '\n'
(gdb) print dbpwd[29]
$15 = 0 '\0'

dbpwd is the base64 encoded password - apparently the password was hashed, then encoded, then the \n stuck on the end, then stored to the database.

But later you say https://bugzilla.redhat.com/show_bug.cgi?id=552421#c11

I removed the trailing 'Cg==' from the userPassword which corresponded to the
newline character and now I can authenticate as the admin user.

So the dbpwd looked like this?

dbpwd[24] == 'C'
dbpwd[25] == 'g'
dbpwd[26] == '='
dbpwd[27] == '='
dbpwd[28] == '\n'
dbpwd[29] == '\0'

If this is the case, it looks like this is what happened:
The password was hashed, then a newline was added, then it was base64 encoded, then another newline was added.

Comment 14 Orion Poplawski 2010-01-08 21:51:28 UTC
Your last summary is correct, but dbpwd is somewhat confused.

userPassword:: in the ldap database was something like:

'e1XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXCg=='

decoded it became:

'{SHA}XXXXXXXXXXXXXXXXXXXXXXXXXX0=\n'

with the newline character there.

dbpwd[26] == '0'
dbpwd[27] == '='
dbpwd[28] == '\n'
dbpwd[29] == '\0'

>>> base64.b64encode('\n')
'Cg=='

Comment 15 Rich Megginson 2010-01-08 22:16:16 UTC
Ok - I thought you meant dbpwd inside the debugger.  The variable in sha_pw_cmp() used to hold the database password is dbpwd.  It's confusing because ldapsearch always base64 encodes the userPassword attribute, even though it's already base64 encoded.

So the problem is that the password was hashed then added to the database like this:
userPassword: "{SHA}" + <b64encoded(hash)> + "\n"

Comment 16 Rich Megginson 2010-01-11 20:14:32 UTC
Created attachment 383080 [details]
patch

Comment 17 Rich Megginson 2010-01-12 03:18:38 UTC
To ssh://git.fedorahosted.org/git/389/ds.git
   786c740..2521eb7  master -> master

commit 2521eb7a649fab115366a652d2ec499ad205ae03
Author: Rich Megginson <rmeggins@redhat.com>
Date:   Mon Jan 11 11:51:39 2010 -0700

    Workaround bogus base64 encoded passwords that end in newline
    
    https://bugzilla.redhat.com/show_bug.cgi?id=552421
    Resolves: bug 552421
    Bug Description: Cannot log into admin server after upgrade (fedora-ds-admin
    Reviewed by: nkinder (Thanks!)
    Branch: HEAD
    Fix Description: Some older versions of setup encoded the admin password in 
    newline character in the dbpwd.  newline is not a valid base64 character.
    Platforms tested: RHEL5 x86_64
    Flag Day: no
    Doc impact: no

Comment 18 Orion Poplawski 2010-03-10 22:43:27 UTC
This is working for me, though I think I've fixed my database so I'm not a good tester anymore.

Comment 19 Amita Sharma 2011-05-11 05:33:27 UTC
Hi Rich,

Request you to please guide me on how should I reproduce it with RHDS.

Thanks,
Amita

Comment 20 Rich Megginson 2011-05-11 13:11:51 UTC
(In reply to comment #19)
> Hi Rich,
> 
> Request you to please guide me on how should I reproduce it with RHDS.
> 
> Thanks,
> Amita

There is nothing to verify with 389 at this time.  We will test this as part of rhds to 389 migration testing.

Comment 21 Rich Megginson 2011-08-31 17:43:12 UTC
Amita, since you were able to upgrade from 8.2 to 9.0 successfully, you can mark this as VERIFIED.

Comment 22 Amita Sharma 2011-08-31 17:48:58 UTC
ok, marking as VERIFIED.


Note You need to log in before you can comment on or make changes to this bug.