Bug 1421869 - Unable to re-add broken AD trust - Unexpected Information received [NEEDINFO]
Summary: Unable to re-add broken AD trust - Unexpected Information received
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: ipa
Version: 7.3
Hardware: x86_64
OS: Linux
Target Milestone: rc
: ---
Assignee: IPA Maintainers
QA Contact: Kaleem
Depends On:
Blocks: 1627814
TreeView+ depends on / blocked
Reported: 2017-02-13 22:26 UTC by Geoff Gatward
Modified: 2018-10-01 12:55 UTC (History)
9 users (show)

Clone Of:
Last Closed: 2018-04-10 16:40:25 UTC
fhanzelk: needinfo? (abokovoy)

Attachments (Terms of Use)
httpd error_log (102.57 KB, text/plain)
2017-02-13 22:26 UTC, Geoff Gatward
no flags Details
Samba log (228.76 KB, text/plain)
2017-02-13 22:26 UTC, Geoff Gatward
no flags Details
proposed patch (1.45 KB, patch)
2017-02-14 10:38 UTC, Alexander Bokovoy
no flags Details | Diff
error_log after applying patch (107.07 KB, text/plain)
2017-02-14 20:25 UTC, Geoff Gatward
no flags Details
updated dcerpc.py (71.44 KB, text/plain)
2017-02-14 20:51 UTC, Alexander Bokovoy
no flags Details
error log after latest dcerpc.py change (113.09 KB, text/plain)
2017-02-16 06:16 UTC, Geoff Gatward
no flags Details

External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:0918 None None None 2018-04-10 16:41 UTC

Description Geoff Gatward 2017-02-13 22:26:07 UTC
Created attachment 1250036 [details]
httpd error_log

I am having a problem with my lab setup (replicates a customer env) in that I seem to have a problem with the trust established to the AD servers.  This was previously set up as a two-way trust on ipa 4.3 and was working - I cannot pinpoint exactly when it broke, but now running ipa 4.4 (AD is 2012r2)

Problem manifests itself in that users in the AD can no longer authenticate to IdM resources.

[root@auth1 ~]# ipa trust-show ad.home.gatwards.org
  Realm name: ad.home.gatwards.org
  Domain NetBIOS name: AD
  Domain Security Identifier: S-1-5-21-2491084655-2534020579-1020697545
  Trust direction: Two-way trust
  Trust type: Active Directory domain

On the AD side,  nltest /domain_trusts /v   shows a Direct Inbound trust for my IdM realm

However I am unable to list the trusted domains:
[root@auth1 ~]# ipa trust-fetch-domains ad.home.gatwards.org
ipa: ERROR: AD domain controller complains about communication sequence. It may mean unsynchronized time on both sides, for example

And trying to re-add (refresh) the trust throws internal error:
[root@auth1 ~]# ipa trust-add --type ad --two-way true ad.home.gatwards.org --admin Administrator
Active Directory domain administrator's password: 
ipa: ERROR: an internal error has occurred

As discussed with Alexander Bokovoy, I removed the trusts from the AD side (mmc snap-in) and tried again with the same result, catching debug logs of the issue.

Comment 1 Geoff Gatward 2017-02-13 22:26 UTC
Created attachment 1250037 [details]
Samba log

Comment 3 Petr Vobornik 2017-02-14 08:42:01 UTC
Upstream ticket:

Comment 4 Alexander Bokovoy 2017-02-14 10:38 UTC
Created attachment 1250170 [details]
proposed patch

Geoff, could you please try this patch against your system?

1. Download the patch, say, fix.patch
2. cd /usr/lib/python2.7/site-packages/
3. patch -p1 <fix.patch
4. restart httpd, systemctl restart httpd
5. Re-establish trust.

Comment 6 Geoff Gatward 2017-02-14 20:24:26 UTC
Unfortunately that didn't do the trick...  

[root@auth1 site-packages]# patch -p1 < /tmp/fix.patch 
patching file ipaserver/dcerpc.py
[root@auth1 site-packages]# systemctl restart httpd
[root@auth1 site-packages]# kinit admin
Password for admin@HOME.GATWARDS.ORG: 
[root@auth1 site-packages]# ipa trust-show ad.home.gatwards.org
ipa: ERROR: ad.home.gatwards.org: trust not found
[root@auth1 site-packages]# ipa trust-add --type=ad ad.home.gatwards.org --admin Administrator --two-way=true
Active Directory domain administrator's password: 
ipa: ERROR: an internal error has occurred

Comment 7 Geoff Gatward 2017-02-14 20:25 UTC
Created attachment 1250328 [details]
error_log after applying patch

Comment 8 Alexander Bokovoy 2017-02-14 20:49:52 UTC
Thanks, this is what I worried. It seems we hit following error condition from MS-LSAD " Forest Trust Collision Generation" section:

 * A top-level name must not be superior to an enabled top-level name for another trusted domain object, unless the current trusted domain object has a corresponding exclusion record.

However, I'm puzzled why we get NT_STATUS_INVALID_PARAMETER rather than collision information.

For a second test, could you please try the following? Just copy attached dcerpc.py over the one in /usr/lib/python2.7/site-packages/ipaserver/. It changes ordering of the exclusion entry we send -- first we would send exclusion entry and then actual TLN for us, in case the order makes difference for Windows Server.

I expect this will not be working either, but before I move to a heavier fix, it is worth trying to re-order entries first.

Comment 9 Alexander Bokovoy 2017-02-14 20:51 UTC
Created attachment 1250341 [details]
updated dcerpc.py

Updated dcerpc.py, just overwrite the old one.

Comment 10 Geoff Gatward 2017-02-16 06:16:04 UTC
Tested with the update dcerpc.py - still fails as you suspected.  Attaching log.

So the issue is that the IPA domain was created first and then the AD domain was an add-on in this environment and AD does not like to be a subdomain of the IPA domain?   I guess in most enterprise environments IPA will be an add-on to existing AD domains which explains why this hasn't been an issue before...

Comment 11 Geoff Gatward 2017-02-16 06:16 UTC
Created attachment 1250784 [details]
error log after latest dcerpc.py change

Comment 12 Alexander Bokovoy 2017-02-16 06:46:00 UTC
Yes, this seems to be the case. Ok, looks like a fix will be more involved.

Comment 14 Alexander Bokovoy 2017-10-19 10:53:49 UTC

I submitted a proposed fix as https://github.com/freeipa/freeipa/pull/1179. This fix filters out subdomains of our primary forest root domain because those would already be covered by *.<forest root> top level name in the topology we send to AD.

Comment 15 Petr Vobornik 2017-12-12 17:23:10 UTC
    443ecbc adtrust: filter out subdomains when defining our topology to AD
    704cf5d adtrust: filter out subdomains when defining our topology to AD
    1490e9c adtrust: filter out subdomains when defining our topology to AD

Comment 18 Ganna Kaihorodova 2018-01-15 14:32:06 UTC
Following conversation in irc the issue that is fixed in the patch is following:
We have 'ipa realmdomains' where we record DNS domains that belong to our realm. When we add a DNS subdomain, we add it to realmdomains too. So when trust is established, we tell AD DCs that these subdomains belong to our realm so that AD knows who is responsible for Kerberos authentication for machines in those DNS subdomains. So if we have ipa.example.com and then add DNS domain ipa2.example.com, AD knows they both are part of IPA.EXAMPLE.COM realm. We add these DNS subdomains also when we create sub.ipa.example.com. From AD perspective sub.ipa.example.com is already covered by ipa.example.com Kerberos route so it causes an error. 

According to that steps for verification are:

1. Install ipa.example.com
2. Add DNS zone ipa2.example.com
3. Create a host machine.ipa2.example.com
4. Create DNS zone ipa3.ipa.example.com
5. Create a host machine.ipa3.ipa.example.com
6. Add a trust to AD forest
7. Check as AD users, kinit user@AD.DOMAIN, kvno -S host machine.ipa2.example.com 
Expected result: it should give ticket to host/machine.ipa2.example.com@IPA.EXAMPLE.COM
8. Check as AD users, kinit user@AD.DOMAIN, kvno -S host machine.ipa3.ipa.example.com 
Expected result:it should give ticket to host/machine.ipa3.ipa.example.com@IPA.EXAMPLE.COM

Comment 22 errata-xmlrpc 2018-04-10 16:40:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.