Bug 1465463 - Active Directory - Unexpected comma or semicolon found at the end of the DN string
Active Directory - Unexpected comma or semicolon found at the end of the DN s...
Status: CLOSED CURRENTRELEASE
Product: ovirt-engine-extension-aaa-ldap
Classification: oVirt
Component: Setup (Show other bugs)
1.3.1
x86_64 Linux
unspecified Severity high (vote)
: ovirt-4.1.8
: 1.3.6
Assigned To: Ondra Machacek
Gonza
: Reopened
Depends On:
Blocks: 1511120
  Show dependency treegraph
 
Reported: 2017-06-27 09:32 EDT by Charlie
Modified: 2017-12-11 11:31 EST (History)
7 users (show)

See Also:
Fixed In Version: ovirt-engine-extension-aaa-ldap-1.3.6
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
OS: CentOS Linux release 7.3.1611 (Core) Relevant Packages: ovirt-engine-extension-aaa-ldap-1.3.1-1.el7.centos.noarch ovirt-engine-4.1.2.2-1.el7.centos.noarch Repo: http://resources.ovirt.org/pub/yum-repo/mirrorlist-ovirt-4.1-el$releasever
Last Closed: 2017-12-11 11:31:51 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Infra
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
rule-engine: ovirt‑4.1+


Attachments (Terms of Use)
Output from Terminal during config. Includes Errors (14.32 KB, text/plain)
2017-06-27 09:32 EDT, Charlie
no flags Details
Fine Logging from ovirt-engine-aaa (231.51 KB, text/plain)
2017-07-31 13:07 EDT, Charlie
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 83790 None None None 2017-11-08 10:34 EST

  None (edit)
Description Charlie 2017-06-27 09:32:53 EDT
Created attachment 1292336 [details]
Output from Terminal during config. Includes Errors

Description of problem:

Configuration is successful with AD, however Login/Test shows exception: "Unexpected comma or semicolon found at the end of the DN string"


Version-Release number of selected component (if applicable):
1.3.1

How reproducible:
Execute the helper tool, "ovirt-engine-extension-aaa-ldap-setup"
Choosing Active Directory, with a resolvable global catalog via forest name, and properly installed System TLS CA for LDAPS server issuer, proper bind DN and password, result in a failed login sequence test.


Steps to Reproduce:
1. Execute Script
2. 3 - Active Directory (Choice)
3. Please enter Active Directory Forest name: corp.mycompany.com
4. Please select protocol to use (startTLS, ldaps, plain) [startTLS]: ldaps
5. Enter search user DN (for example uid=username,dc=example,dc=com or leave empty for anonymous): CN=ottch,OU=US,OU=Users,OU=Corporate,DC=corp,DC=mycompany,DC=com
6. Enter search user password: ****************
7. Are you going to use Single Sign-On for Virtual Machines (Yes, No) [No]: No
8. Please specify profile name that will be visible to users [corp.mycompany.com]: corp.mycompany.com
9. Select test sequence to execute (Done, Abort, Login, Search) [Abort]: Login


Actual results:
2017-06-27 12:33:48,974Z WARNING [ovirt-engine-extension-aaa-ldap.authn::corp.mycompany.com-authn] Cannot initialize LDAP framework, deferring initialization. Error: Unexpected comma
or semicolon found at the end of the DN string.
          2017-06-27 12:33:48,975Z SEVERE  Unexpected comma or semicolon found at the end of the DN string.
[ ERROR ] Login sequence failed

Expected results:
Login Test Successful

Additional info:
Not sure what the severity is for others, but I'm marking it as "high" due to the fact I cannot configure authentication for my 9 node ovirt cluster and have 15 developers that need SSO for the ovirt portal.
Comment 1 Charlie 2017-06-27 14:53:40 EDT
Possibly related, but I don't think so.  The account password has a ! character.  I have seen issues similar with JBOSS/Wildfly, e.g. https://access.redhat.com/solutions/1499143
Don't think this is the issue, but if having trouble reproducing I would say to test using special character in the password field?

Also I noticed the DN is much longer than the example one.  The exception has me wondering if there is a parsing issue with the DN I have provided for the search account?

Thanks in advance,

Charlie
Comment 2 Charlie 2017-06-28 10:53:30 EDT
I have decided to add the following information as additional reference:

[root@devops ~]# ldapsearch -x -D "ottch@mycompany.com" -H ldaps://ldap-corp.mycompany.com -b "OU=US,OU=Users,OU=Corporate,DC=corp,DC=mycompany,DC=com" -W "(uid=ottch)"
Enter LDAP Password: 

# extended LDIF
#
# LDAPv3
# base <OU=US,OU=Users,OU=Corporate,DC=corp,DC=mycompany,DC=com> with scope subtree
# filter: (uid=ottch)
# requesting: ALL
#

# ottch, US, Users, Corporate, corp.mycompany.com
dn: CN=ottch,OU=US,OU=Users,OU=Corporate,DC=corp,DC=mycompany,DC=com
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: user
cn: ottch
sn: Ott
c: US
l: Charlottesville
st: VA
o: US
title: Systems Engineer, Lead
description: Password Changed 05/30/2017 09:03:38 (MMP)
[Rest of data omitted intentionally...]


The interesting thing here is that I am using the LDAP server "ldap-corp.mycompany.com" which AD administrators have told us to use for LDAP.  This is currently working with a few other Apps, e.g. Foreman LDAP login...

However when following the output I attached to this ticket, i see requests to AUTHP-AD-CORP22.corp.mycompany.com:389 which are reverse dns lookups for the IPs resolved from the round-robin 'corp.mycompany.com' (Forest Name?)

One issue I'm not clear on is domain name vs. forest name in this context?  When authentication to online services I normally use, "mycompany-corp\username", which is a short name for the domain?  I *think* the forest should just be mycompany.com but the setup script fails when I use that.  Instead I have to use corp.mycompany.com.  Perhaps the AD Schema is not what you expect in the setup script?
Comment 3 Ondra Machacek 2017-07-18 04:52:06 EDT
(In reply to Charlie from comment #2)
> However when following the output I attached to this ticket, i see requests
> to AUTHP-AD-CORP22.corp.mycompany.com:389 which are reverse dns lookups for
> the IPs resolved from the round-robin 'corp.mycompany.com' (Forest Name?)

When you insert 'corp.mycompany.com' as a forest name, what we do is that we look for global catalogs and domains, by looking for DNS SRV _ldap._tcp records.

Your DNS records have configured those servers on port 389, but you want port 636. So you can create a records in your DNS for example _ldaps._tcp with 636 port, but what I've seen most people don't want to do that.

So may I ask you why you don't use just startTLS on port 389? You issue is that it tries to do ldaps connection on 389, which fails.

> 
> One issue I'm not clear on is domain name vs. forest name in this context? 
> When authentication to online services I normally use,
> "mycompany-corp\username", which is a short name for the domain?  I *think*
> the forest should just be mycompany.com but the setup script fails when I
> use that.  Instead I have to use corp.mycompany.com.  Perhaps the AD Schema
> is not what you expect in the setup script?

How exactly it fails? Domain vs forest name is exactly how Microsoft is documenting it, as we automatically resolves all domains in a forest, we work with the forest name, you can later use all domains in that forest, not just one.

If you insist and you want to work only with a list of some domains, not whole forest you can check this bugzilla:

  https://bugzilla.redhat.com/show_bug.cgi?id=1438319
Comment 4 Charlie 2017-07-31 11:48:34 EDT
I have switched utilizing startTLS with 389, and it seems that search is successful.  However I cannot get "Login" to work.

Exception:

          2017-07-31 15:38:48,146Z SEVERE  Unexpected comma or semicolon found at the end of the DN string.
[ ERROR ] Login sequence failed
          Please investigate details of the failure (search for lines containing SEVERE log level).


I have tried both my username/uid "ottch" as well as "ottch@mycompany.com"

Both result in the same SEVERE error that there was an unexpected comma or semicolon found at the end of the DN string.

Looking at the generated files,
/etc/ovirt-engine/aaa/corp.leidos.com.properties
/etc/ovirt-engine/extensions.d/corp.mycompany.com-authz.properties
/etc/ovirt-engine/extensions.d/corp.mycompany.com-authn.properties

I only see the vars.user defined and the other settings seem to only reference the global:vars.user or global:vars.domain.  So I am unsure how the Login DN is generated.
Comment 5 Ondra Machacek 2017-07-31 12:58:59 EDT
Can you please share the FINEST log:

 $ ovirt-engine-extensions-tool --log-level=FINEST --log-file=/tmp/aaa.log aaa login-user --user-name=ottch@mycompany.com --profile=corp.mycompany.com
Comment 6 Charlie 2017-07-31 13:07 EDT
Created attachment 1307213 [details]
Fine Logging from ovirt-engine-aaa
Comment 7 Charlie 2017-07-31 13:08:28 EDT
This part seems troublesome...

FINE: getConnectionPoolEntry Entry name='authn', dn='null'
Jul 31, 2017 5:01:57 PM org.ovirt.engineextensions.aaa.ldap.Framework authCheck
FINE: Authentication exception
java.lang.NullPointerException
        at org.ovirt.engineextensions.aaa.ldap.Framework.authCheck(Framework.java:914)
        at org.ovirt.engineextensions.aaa.ldap.Framework.runSequence(Framework.java:1472)
        at org.ovirt.engineextensions.aaa.ldap.AuthnExtension.doAuthenticateCredentials(AuthnExtension.java:152)



Full log attached to report
Comment 8 Charlie 2017-07-31 13:18:50 EDT
Noticed many TLS failures, I've installed the CA for the 636 LDAPS issued cert, but perhaps the starttls 389 certificate has a different CA...
Comment 9 Ondra Machacek 2017-08-01 05:29:45 EDT
No, you have just some of the AD with wrong certificate. For example AUTHP-AD-CORP02 is fine, but AUTHP-AD-UK06 and AUTHP-AD-CORP19 are not.

But still we shouldn't fail with NPE, but with some reasonable error.
Comment 10 Charlie 2017-08-01 12:01:27 EDT
I think that makes sense, looking at the cert for UK06 and CORP19 they seem to have a newer issuer...

Do you think the NPE is a result of my input during configuration or rather, something that is not resolving via directory service lookup?

I downloaded Apache Directory Studio and created a StartTLS connection to one of the servers on port 389.  I found that I was able to authenticate with two different user name patterns...

1.) ottch@mycompany.com
2.) cn=ottch,OU=US,OU=Users,OU=Corporate,DC=corp,DC=mycompany,DC=com

Both of the entries above allowed successful connection.
Comment 11 Charlie 2017-08-01 12:10:15 EDT
Some more config info:

[root@pedro-godines aaa]# ls
corp.mycompany.com.properties  internal.properties

[root@pedro-godines aaa]# cat corp.mycompany.com.properties 
include = <ad.properties>

vars.domain = corp.mycompany.com
vars.user = cn=ottch,OU=US,OU=Users,OU=Corporate,DC=corp,DC=mycompany,DC=com
vars.password = ****************

pool.default.auth.simple.bindDN = ${global:vars.user}
pool.default.auth.simple.password = ${global:vars.password}
pool.default.serverset.type = srvrecord
pool.default.serverset.srvrecord.domain = ${global:vars.domain}
pool.default.ssl.startTLS = true

[root@pedro-godines aaa]# cat internal.properties 
config.datasource.jdbcurl=jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory
config.datasource.dbuser=engine
config.datasource.dbpassword=**********************
config.datasource.jdbcdriver=org.postgresql.Driver
config.datasource.schemaname=aaa_jdbc

[root@pedro-godines aaa]# pwd
/etc/ovirt-engine/aaa
Comment 12 Gonza 2017-11-24 11:51:54 EST
Verified with:
ovirt-engine-extension-aaa-ldap-setup-1.3.6-1.el7ev.noarch

Login sequence executed successfully
Comment 13 Steve D 2017-12-08 11:09:22 EST
I'm running rhv 4.1 and just ran into the same issue after about 2 months of using rhv and restarting the manager VM. My AD bind password contained an '!', changing that character to a '$' and restarting ovirt-engine service resolved the issue.

2017-12-08 07:49:46,931-08 ERROR [org.ovirt.engine.core.aaa.servlet.SsoPostLoginServlet] (default task-12) [] server_error: Unexpected comma
 or semicolon found at the end of the DN string.
Comment 14 Martin Perina 2017-12-11 03:55:56 EST
(In reply to Steve D from comment #13)
> I'm running rhv 4.1 and just ran into the same issue after about 2 months of
> using rhv and restarting the manager VM. My AD bind password contained an
> '!', changing that character to a '$' and restarting ovirt-engine service
> resolved the issue.
> 
> 2017-12-08 07:49:46,931-08 ERROR
> [org.ovirt.engine.core.aaa.servlet.SsoPostLoginServlet] (default task-12) []
> server_error: Unexpected comma
>  or semicolon found at the end of the DN string.

Please check certificates on all of your AD servers which may be used by this domain configuration, at least one of such servers has invalid certificate.
Fix showing more understandable message will be part of 4.1.8 release.
Comment 15 Ondra Machacek 2017-12-11 04:03:49 EST
(In reply to Steve D from comment #13)
> I'm running rhv 4.1 and just ran into the same issue after about 2 months of
> using rhv and restarting the manager VM. My AD bind password contained an
> '!', changing that character to a '$' and restarting ovirt-engine service
> resolved the issue.

Your issue is different that this bugzilla. Your issue seems to be duplicate of bug 1443645, it should be fixed in version 4.2.

Note You need to log in before you can comment on or make changes to this bug.